{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "8fd9a764-db4c-41f7-94f2-5fb78542fcca",
   "metadata": {
    "id": "8fd9a764-db4c-41f7-94f2-5fb78542fcca"
   },
   "source": [
    "# Quick Overview of RLFH\n",
    "\n",
    "The performance of Language Models until GPT-3 was kind of amazing as-is. What the models of were essentially lacking was the aspect of **alignment**. The language generation aspect was particularly challenging due to heavy hallucinations, toxicity, etc.\n",
    "\n",
    "The authors of the seminal work **[InstructGPT](https://arxiv.org/pdf/2203.02155)** basically focussed on this aspect of aligning the language models to user's intructions (hence the name!). Their work showcased how we can further fine-tune such models in a supervised way leverage human feedback and reinforcement learning to align them.\n",
    "\n",
    "## High-Level Overview of the Setup\n",
    "\n",
    "<img src=\"./assets/instruct_gpt_rlhf.png\">\n",
    "\n",
    "> Source: https://arxiv.org/pdf/2203.02155"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c8a0b86-baa5-49fd-b5b2-cb8a93aeacbc",
   "metadata": {
    "id": "8c8a0b86-baa5-49fd-b5b2-cb8a93aeacbc"
   },
   "source": [
    "## Key Concepts:\n",
    "\n",
    "- **Reinforcement Learning (RL)**: A machine learning paradigm where an agent learns to make decisions by performing actions and receiving _rewards or penalties_ .\n",
    "- **Human Feedback**: Evaluations provided by humans that guide the learning process, ensuring the model's outputs align with human expectations and preferences."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe465e94-5116-4ada-8b4b-bcc2aaffe6d1",
   "metadata": {
    "id": "fe465e94-5116-4ada-8b4b-bcc2aaffe6d1"
   },
   "source": [
    "## How Does this Actually Work? Show Me Examples Please?\n",
    "\n",
    "### Standard Supervised Learning:\n",
    "- Input: \"Generate a story about a dragon and a knight.\"\n",
    "- Output: The model generates a story based on its training data.\n",
    "\n",
    "\n",
    "### Reinforcement Learning From Human Feedback:\n",
    "- Input: \"Generate a story about a dragon and a knight.\"\n",
    "- Initial Output: The model generates a story.\n",
    "- Human Feedback: A human rates the story on coherence, creativity, and engagement.\n",
    "- Adjusted Output: The model refines its story generation based on the feedback, leading to more engaging and coherent stories over time.\n",
    "\n",
    "As pointed out in the figure above, one of the ways of bringing this alignment is through:\n",
    "- Training a **reward model** using a Human labelled dataset.\n",
    "    - This dataset basically contains rank ordered responses to different inputs to the model\n",
    "- The reward model learns to predict human preferences based on the provided human feedback by assigning a score (reward) to the outputs of the language model.\n",
    "-  The output of the reward model is then used to update the policy of the language model (agent) to align with the human feedback (exploration vs exploitation)\n",
    "\n",
    "\n",
    "<img src=\"https://huggingface.co/datasets/trl-internal-testing/example-images/resolve/main/images/trl_overview.png\">\n",
    "\n",
    "> Source: https://huggingface.co/docs/trl/en/quickstart"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0670ad0a-0feb-4893-94ef-0582aa14c0a1",
   "metadata": {
    "id": "0670ad0a-0feb-4893-94ef-0582aa14c0a1"
   },
   "source": [
    "## PPO vs DPO\n",
    "\n",
    "There are different ways of performing policy optimisation. The original work follows Proximal Policy Optimisation which requires a separate reward model to tune the Language Model. DPO or Direct Policy Optimisation directly applies updates to the language model thus removing the need for a separate reward model.\n",
    "\n",
    "> Read more about RL and KL Divergence to understand the topic better\n",
    "\n",
    "\n",
    "> The KL-divergence between the two outputs is used as an _additional reward signal_ to make sure the generated responses don’t deviate too far from the reference language model."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0895576f-f530-4268-b13d-a2898c160c6d",
   "metadata": {
    "id": "0895576f-f530-4268-b13d-a2898c160c6d"
   },
   "source": [
    "# Quick Hello World Using PPO\n",
    "\n",
    "> Can we improve alignment of gpt2?\n",
    "\n",
    "> Adapted from:\n",
    "> [[1](https://huggingface.co/docs/trl/en/quickstart)], [[2](https://github.com/antndlcrx/oxford-llms-workshop/blob/main/materials/seminars/day_3/8_LLMs%20alignment%20with%20RLHF.ipynb)]\n",
    "\n",
    "**NOTE**: Ensure you have high-capacity GPU available for this notebook!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9f4c1c30-1627-4373-affb-6ae932062c31",
   "metadata": {
    "id": "9f4c1c30-1627-4373-affb-6ae932062c31"
   },
   "source": [
    "## Install Dependencies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "f1aac1de-86c7-41f6-8bcf-0871c8e7e75c",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "f1aac1de-86c7-41f6-8bcf-0871c8e7e75c",
    "outputId": "bef8b639-bcf0-47b4-a25e-ae8230f8f214",
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: transformers in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (4.44.2)\n",
      "Collecting trl\n",
      "  Downloading trl-0.10.1-py3-none-any.whl.metadata (12 kB)\n",
      "Requirement already satisfied: datasets in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (2.21.0)\n",
      "Requirement already satisfied: accelerate in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (0.33.0)\n",
      "Requirement already satisfied: matplotlib in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (3.9.1)\n",
      "Collecting tensorboard\n",
      "  Downloading tensorboard-2.17.1-py3-none-any.whl.metadata (1.6 kB)\n",
      "Collecting tensorboardx\n",
      "  Downloading tensorboardX-2.6.2.2-py2.py3-none-any.whl.metadata (5.8 kB)\n",
      "Collecting seaborn\n",
      "  Downloading seaborn-0.13.2-py3-none-any.whl.metadata (5.4 kB)\n",
      "Requirement already satisfied: filelock in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from transformers) (3.15.4)\n",
      "Requirement already satisfied: huggingface-hub<1.0,>=0.23.2 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from transformers) (0.24.1)\n",
      "Requirement already satisfied: numpy>=1.17 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from transformers) (1.26.4)\n",
      "Requirement already satisfied: packaging>=20.0 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from transformers) (24.1)\n",
      "Requirement already satisfied: pyyaml>=5.1 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from transformers) (6.0.1)\n",
      "Requirement already satisfied: regex!=2019.12.17 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from transformers) (2024.5.15)\n",
      "Requirement already satisfied: requests in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from transformers) (2.32.3)\n",
      "Requirement already satisfied: safetensors>=0.4.1 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from transformers) (0.4.3)\n",
      "Requirement already satisfied: tokenizers<0.20,>=0.19 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from transformers) (0.19.1)\n",
      "Requirement already satisfied: tqdm>=4.27 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from transformers) (4.66.4)\n",
      "Requirement already satisfied: torch>=1.4.0 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from trl) (2.3.1)\n",
      "Collecting tyro>=0.5.11 (from trl)\n",
      "  Downloading tyro-0.8.10-py3-none-any.whl.metadata (8.4 kB)\n",
      "Requirement already satisfied: pyarrow>=15.0.0 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from datasets) (17.0.0)\n",
      "Requirement already satisfied: dill<0.3.9,>=0.3.0 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from datasets) (0.3.8)\n",
      "Requirement already satisfied: pandas in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from datasets) (2.2.2)\n",
      "Requirement already satisfied: xxhash in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from datasets) (3.4.1)\n",
      "Requirement already satisfied: multiprocess in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from datasets) (0.70.16)\n",
      "Requirement already satisfied: fsspec<=2024.6.1,>=2023.1.0 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from fsspec[http]<=2024.6.1,>=2023.1.0->datasets) (2024.5.0)\n",
      "Requirement already satisfied: aiohttp in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from datasets) (3.9.5)\n",
      "Requirement already satisfied: psutil in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from accelerate) (6.0.0)\n",
      "Requirement already satisfied: contourpy>=1.0.1 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from matplotlib) (1.2.1)\n",
      "Requirement already satisfied: cycler>=0.10 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from matplotlib) (0.12.1)\n",
      "Requirement already satisfied: fonttools>=4.22.0 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from matplotlib) (4.53.1)\n",
      "Requirement already satisfied: kiwisolver>=1.3.1 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from matplotlib) (1.4.5)\n",
      "Requirement already satisfied: pillow>=8 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from matplotlib) (10.4.0)\n",
      "Requirement already satisfied: pyparsing>=2.3.1 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from matplotlib) (3.1.2)\n",
      "Requirement already satisfied: python-dateutil>=2.7 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from matplotlib) (2.9.0.post0)\n",
      "Collecting absl-py>=0.4 (from tensorboard)\n",
      "  Downloading absl_py-2.1.0-py3-none-any.whl.metadata (2.3 kB)\n",
      "Collecting grpcio>=1.48.2 (from tensorboard)\n",
      "  Downloading grpcio-1.66.1-cp311-cp311-macosx_10_9_universal2.whl.metadata (3.9 kB)\n",
      "Collecting markdown>=2.6.8 (from tensorboard)\n",
      "  Downloading Markdown-3.7-py3-none-any.whl.metadata (7.0 kB)\n",
      "Collecting protobuf!=4.24.0,>=3.19.6 (from tensorboard)\n",
      "  Downloading protobuf-5.28.1-cp38-abi3-macosx_10_9_universal2.whl.metadata (592 bytes)\n",
      "Requirement already satisfied: setuptools>=41.0.0 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from tensorboard) (65.5.0)\n",
      "Requirement already satisfied: six>1.9 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from tensorboard) (1.16.0)\n",
      "Collecting tensorboard-data-server<0.8.0,>=0.7.0 (from tensorboard)\n",
      "  Downloading tensorboard_data_server-0.7.2-py3-none-any.whl.metadata (1.1 kB)\n",
      "Collecting werkzeug>=1.0.1 (from tensorboard)\n",
      "  Downloading werkzeug-3.0.4-py3-none-any.whl.metadata (3.7 kB)\n",
      "Requirement already satisfied: aiosignal>=1.1.2 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from aiohttp->datasets) (1.3.1)\n",
      "Requirement already satisfied: attrs>=17.3.0 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from aiohttp->datasets) (23.2.0)\n",
      "Requirement already satisfied: frozenlist>=1.1.1 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from aiohttp->datasets) (1.4.1)\n",
      "Requirement already satisfied: multidict<7.0,>=4.5 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from aiohttp->datasets) (6.0.5)\n",
      "Requirement already satisfied: yarl<2.0,>=1.0 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from aiohttp->datasets) (1.9.4)\n",
      "Requirement already satisfied: typing-extensions>=3.7.4.3 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from huggingface-hub<1.0,>=0.23.2->transformers) (4.12.2)\n",
      "Requirement already satisfied: pytz>=2020.1 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from pandas->datasets) (2024.1)\n",
      "Requirement already satisfied: tzdata>=2022.7 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from pandas->datasets) (2024.1)\n",
      "Requirement already satisfied: charset-normalizer<4,>=2 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from requests->transformers) (3.3.2)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from requests->transformers) (3.7)\n",
      "Requirement already satisfied: urllib3<3,>=1.21.1 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from requests->transformers) (2.2.2)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from requests->transformers) (2024.7.4)\n",
      "Requirement already satisfied: sympy in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from torch>=1.4.0->trl) (1.13.1)\n",
      "Requirement already satisfied: networkx in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from torch>=1.4.0->trl) (3.3)\n",
      "Requirement already satisfied: jinja2 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from torch>=1.4.0->trl) (3.1.4)\n",
      "Collecting docstring-parser>=0.16 (from tyro>=0.5.11->trl)\n",
      "  Downloading docstring_parser-0.16-py3-none-any.whl.metadata (3.0 kB)\n",
      "Collecting rich>=11.1.0 (from tyro>=0.5.11->trl)\n",
      "  Downloading rich-13.8.1-py3-none-any.whl.metadata (18 kB)\n",
      "Collecting shtab>=1.5.6 (from tyro>=0.5.11->trl)\n",
      "  Downloading shtab-1.7.1-py3-none-any.whl.metadata (7.3 kB)\n",
      "Requirement already satisfied: MarkupSafe>=2.1.1 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from werkzeug>=1.0.1->tensorboard) (2.1.5)\n",
      "Collecting markdown-it-py>=2.2.0 (from rich>=11.1.0->tyro>=0.5.11->trl)\n",
      "  Downloading markdown_it_py-3.0.0-py3-none-any.whl.metadata (6.9 kB)\n",
      "Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from rich>=11.1.0->tyro>=0.5.11->trl) (2.18.0)\n",
      "Requirement already satisfied: mpmath<1.4,>=1.1.0 in /Users/raghavbali/.pyenv/versions/3.11.9/envs/deeplearning/lib/python3.11/site-packages (from sympy->torch>=1.4.0->trl) (1.3.0)\n",
      "Collecting mdurl~=0.1 (from markdown-it-py>=2.2.0->rich>=11.1.0->tyro>=0.5.11->trl)\n",
      "  Downloading mdurl-0.1.2-py3-none-any.whl.metadata (1.6 kB)\n",
      "Downloading trl-0.10.1-py3-none-any.whl (280 kB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m280.1/280.1 kB\u001b[0m \u001b[31m4.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m MB/s\u001b[0m eta \u001b[36m0:00:01\u001b[0m\n",
      "\u001b[?25hDownloading tensorboard-2.17.1-py3-none-any.whl (5.5 MB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m5.5/5.5 MB\u001b[0m \u001b[31m40.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m31m43.1 MB/s\u001b[0m eta \u001b[36m0:00:01\u001b[0m\n",
      "\u001b[?25hDownloading tensorboardX-2.6.2.2-py2.py3-none-any.whl (101 kB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m101.7/101.7 kB\u001b[0m \u001b[31m13.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading seaborn-0.13.2-py3-none-any.whl (294 kB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m294.9/294.9 kB\u001b[0m \u001b[31m23.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading absl_py-2.1.0-py3-none-any.whl (133 kB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m133.7/133.7 kB\u001b[0m \u001b[31m12.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading grpcio-1.66.1-cp311-cp311-macosx_10_9_universal2.whl (10.6 MB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m10.6/10.6 MB\u001b[0m \u001b[31m49.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0mm eta \u001b[36m0:00:01\u001b[0m:01\u001b[0m:01\u001b[0m\n",
      "\u001b[?25hDownloading Markdown-3.7-py3-none-any.whl (106 kB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m106.3/106.3 kB\u001b[0m \u001b[31m10.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading protobuf-5.28.1-cp38-abi3-macosx_10_9_universal2.whl (414 kB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m414.7/414.7 kB\u001b[0m \u001b[31m30.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading tensorboard_data_server-0.7.2-py3-none-any.whl (2.4 kB)\n",
      "Downloading tyro-0.8.10-py3-none-any.whl (105 kB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m105.7/105.7 kB\u001b[0m \u001b[31m9.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading werkzeug-3.0.4-py3-none-any.whl (227 kB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m227.6/227.6 kB\u001b[0m \u001b[31m26.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading docstring_parser-0.16-py3-none-any.whl (36 kB)\n",
      "Downloading rich-13.8.1-py3-none-any.whl (241 kB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m241.6/241.6 kB\u001b[0m \u001b[31m21.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading shtab-1.7.1-py3-none-any.whl (14 kB)\n",
      "Downloading markdown_it_py-3.0.0-py3-none-any.whl (87 kB)\n",
      "\u001b[2K   \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m87.5/87.5 kB\u001b[0m \u001b[31m7.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading mdurl-0.1.2-py3-none-any.whl (10.0 kB)\n",
      "Installing collected packages: werkzeug, tensorboard-data-server, shtab, protobuf, mdurl, markdown, grpcio, docstring-parser, absl-py, tensorboardx, tensorboard, markdown-it-py, seaborn, rich, tyro, trl\n",
      "Successfully installed absl-py-2.1.0 docstring-parser-0.16 grpcio-1.66.1 markdown-3.7 markdown-it-py-3.0.0 mdurl-0.1.2 protobuf-5.28.1 rich-13.8.1 seaborn-0.13.2 shtab-1.7.1 tensorboard-2.17.1 tensorboard-data-server-0.7.2 tensorboardx-2.6.2.2 trl-0.10.1 tyro-0.8.10 werkzeug-3.0.4\n",
      "\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m24.0\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.2\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpython3.11 -m pip install --upgrade pip\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "!pip3 install transformers trl datasets accelerate matplotlib  tensorboard tensorboardx seaborn"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8af9fccb-8f74-4055-837a-290bc88d7abc",
   "metadata": {
    "id": "8af9fccb-8f74-4055-837a-290bc88d7abc"
   },
   "source": [
    "## Import Packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "1cb50023-862f-4732-9810-847ed649492a",
   "metadata": {
    "id": "1cb50023-862f-4732-9810-847ed649492a"
   },
   "outputs": [],
   "source": [
    "import random\n",
    "import torch\n",
    "import time\n",
    "import os\n",
    "from tqdm.notebook import tqdm\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from random import choices\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "tqdm.pandas()\n",
    "\n",
    "from datasets import load_dataset, Dataset\n",
    "from transformers import AutoTokenizer, pipeline\n",
    "from trl import PPOTrainer, PPOConfig, AutoModelForCausalLMWithValueHead, create_reference_model\n",
    "from trl.core import LengthSampler"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6df2c94d-bf36-4c16-aa91-d422eed1a076",
   "metadata": {
    "id": "6df2c94d-bf36-4c16-aa91-d422eed1a076"
   },
   "source": [
    "## Configs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "a741d7a7-dc43-45aa-905b-c0950c90efc8",
   "metadata": {
    "id": "a741d7a7-dc43-45aa-905b-c0950c90efc8"
   },
   "outputs": [],
   "source": [
    "text_input_length = 5\n",
    "text_output_length = 20\n",
    "seed = 42\n",
    "\n",
    "sentiment_pipe_kwargs = {\n",
    "    \"top_k\": None,\n",
    "    \"function_to_apply\": \"none\",\n",
    "    'return_all_scores':False\n",
    "}\n",
    "\n",
    "ppo_config = PPOConfig(\n",
    "    model_name=\"raghavbali/gpt2-movie_reviewer\",\n",
    "    steps=200,\n",
    "    learning_rate=1.41e-5,\n",
    "    remove_unused_columns=False,\n",
    "    log_with=\"tensorboard\",\n",
    "    project_kwargs={\"logging_dir\": \"./logs\"},\n",
    ")\n",
    "\n",
    "generation_kwargs = {\n",
    "    \"min_length\": -1,\n",
    "    \"top_k\": 0.0,\n",
    "    \"top_p\": 1.0,\n",
    "    \"do_sample\": True,\n",
    "}\n",
    "\n",
    "ALIGNED_MODEL_NAME = f\"aligned-{ppo_config.model_name.split('/')[1]}\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "c370fbf6-9688-4ef4-8766-e818d50365e1",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 145,
     "referenced_widgets": [
      "b7ae5d5cecee4485b7aaa6fe7571182c",
      "6960797f105246e582de14c69bd88171",
      "089c84d7a2384a99b6ee278722bdb867",
      "80b0498891874b709d8291a47fe70350",
      "6a106ae702b546538de0429353d4ba62",
      "2e753528edde49d8a50c57ef204ac5b3",
      "73ed87030dc24eb681d9ba9d0f4b9f11",
      "e106952a01f54610997d830e471ebda0",
      "9cde8a8dcafe4237bfb5e96075b9f158",
      "05b8baecad7740f5834d453a84e82f4a",
      "97412f765b2942519106c00ffa2020ba",
      "addcfa65505e4679a03c2ef1ceedf8c0",
      "f24072cbb64e4011af425afb51adabfb",
      "526dc2e823a74e71a84cb4b0d50cd5be",
      "29ce0e9095b646c3b50d721559af6bf6",
      "875ffc1310844528b287d4a257cf5f97",
      "7fee544e10ba4b399bae8c1a68c16e5b",
      "c943662f45a34434b2c047a5d1019e04",
      "656bbd9009654ed6af0c61dba2f01d4d",
      "5ecff38cd27f4a1ba36dfaddaaee0bb9",
      "731e42b4b9e3444eadff0c496976389e",
      "94817bcac7374e43b4b633e0f2d13889",
      "dfd2fad3bba04fd295ee408c91869f7c",
      "e84412f70dd049dea7cb079c2c11c3d4",
      "d39ee62683574bd0a02930bef6b94dcc",
      "47c77d548220417abec9f72df138401a",
      "e23b1364318a46f680cabedc7f0c6d30",
      "08ffbe2a042544aeb8565179a327c986",
      "42f8b2a5cc8845c489dadaade07e35df",
      "17c551db5ae944a1ad9fe7fc38f88a9f",
      "fbe86cfbd1d64b02b43adfd5304bca49",
      "d1269dd6eb1e45b1aa1a6589dc9f642f"
     ]
    },
    "id": "c370fbf6-9688-4ef4-8766-e818d50365e1",
    "outputId": "8ebdfb73-6de6-4339-ff74-a311aab7d749"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b7b95501dfb646cd8eff39a64d8f2a42",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(HTML(value='<center> <img\\nsrc=https://huggingface.co/front/assets/huggingface_logo-noborder.sv…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from huggingface_hub import notebook_login\n",
    "\n",
    "notebook_login()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "f3dc48d6-09d0-4a4b-8dd4-bece70b3576e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "You have chosed NOT TO save trained objects to HF, objects will remain on your system \n"
     ]
    }
   ],
   "source": [
    "# if you want to save your trained model objects , set the flag to True\n",
    "# and provide a valid HF namespace/username\n",
    "# Make sure the token you have created has write permissions\n",
    "SAVE_TRAINED_OBJECTS = False\n",
    "HF_NAMESPACE_TO_PUSH_TO = \"XXXXXX\"\n",
    "\n",
    "if SAVE_TRAINED_OBJECTS and HF_NAMESPACE_TO_PUSH_TO != \"XXXXXX\":\n",
    "    print(f\"You have chosen to save trained objects to {HF_NAMESPACE_TO_PUSH_TO} \")\n",
    "else:\n",
    "    print(f\"You have chosed NOT TO save trained objects to HF, objects will remain on your system \")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1421f780-6e7e-4a18-877b-d17d05a55857",
   "metadata": {
    "id": "1421f780-6e7e-4a18-877b-d17d05a55857"
   },
   "source": [
    "## Download Models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "78f832e9-0109-49cd-afce-c2b7940b2081",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 249,
     "referenced_widgets": [
      "f12bf5040447467a882e40fcf3e972d4",
      "f2750715312a40b0ab1ba8c2343f9a98",
      "d9e2f543e5544f85b24fff37f375ae88",
      "945a08de02ee4bef9ecd00e64b582246",
      "5f8193d2901645feb4f3fc80d4942bad",
      "e8022095601b4285aad6dd1a7181cd6d",
      "2ac034bd2c6d44d6aad88f8af57d3f13",
      "f3b3939fded04c1397ebb11010abd533",
      "56c8efa2aff44f90a1a595783d3d7d1c",
      "f7aeb464ce3042e2b0df49978a777a8b",
      "fedfb03cd8b84712b2b10636dc298178",
      "410adfcb106444c4a5d52f9490788e7d",
      "905482147fcc4b0eadad09ec99333097",
      "193ceb339ec1490587881933788b3015",
      "75ada9f5c4d9466e8a42597ea9ac6466",
      "806a776295c44eb7998dca7ace00c6cf",
      "af13120b305c49ddb60dec814bb23ad0",
      "2fb4cb7c8c6f44a2a5cc4f8badde0b9a",
      "cae1ddea6b0b4accb7a5ea9a320070ea",
      "2bb6639cb150487ab78a7b0d2d54a3ff",
      "0de62df79d03478184c5c5413795e479",
      "3c426c71a2d147c1af9330ce0703e14d",
      "8a7ecf987e09457f97a12db63998ed09",
      "843c3e08d63547d9a42eec08ecd8f1ed",
      "4a782c3e43c94067a571c2342bf92165",
      "b2012e7cfe3e408e82a4cf52d4ff06bf",
      "d530e3d8282e4971a94dfbbd47a4f0ff",
      "5950da05ca49401d9a8ec085b5e54fa8",
      "4fddf09bfffa4b379a7a6378d9856b20",
      "55ce1d040d1d496ca8d15e8e67c74555",
      "98fe1d638a9c47a3a29217dec010271f",
      "c8a1a90eb49e487cba872d50213fe0ac",
      "1b9ef18c0fbf4ff39d74e2d54c107991",
      "5b0ee610309d47088af9a940d901038a",
      "ca28a78a4fcf41919f03fab4488e2f10",
      "42cdd539bdcd4563ac7263ca3a87a795",
      "e16b3717548e405ea82b33e6f1ed9aeb",
      "7ff2fe616e7f4890a3d7e5dd5b1539bf",
      "da3a18c49dc54741a40ba5aa204b39b9",
      "09e2170d4b814dec842545ffef6aa186",
      "33a3ca1767f94716bfa274e977c43215",
      "5d75918be7df4027a56f972503bd8b78",
      "1f038ef1ec8b4d1e88654b68178406f4",
      "21168c6da7434941a8062e727a8acb74",
      "649625e0f9be4c9ebb0a119076333197",
      "26438f4a85cb4e05923cbd1444df2364",
      "2cab9533d3c1442a90872341972df272",
      "1c5621316bce482abcd926e7a053a9b0",
      "5cd71a878a2b45a9bdaaf56f32f2da6c",
      "dc0b9472eea14c7bbd285bef3d044583",
      "373c0550a0e641f1a02ae3b4e14440e8",
      "e454660115b240f1aea38f25d04012e0",
      "67e45ac4538d4183b1eea084e899dd48",
      "80f799821f6145238fccc14826e56e8e",
      "f5a4aa9b9275450e8d3f028b5ee4d4ce"
     ]
    },
    "id": "78f832e9-0109-49cd-afce-c2b7940b2081",
    "outputId": "df3551f6-8772-402b-e543-8164d65fdb1c"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:A <class 'transformers.models.gpt2.modeling_gpt2.GPT2LMHeadModel'> model is loaded from 'raghavbali/gpt2-movie_reviewer', and no v_head weight is found. This IS expected if you are not resuming PPO training.\n"
     ]
    }
   ],
   "source": [
    "model = AutoModelForCausalLMWithValueHead.from_pretrained(ppo_config.model_name)\n",
    "# create a reference model\n",
    "ref_model = create_reference_model(model, num_shared_layers=6)\n",
    "tokenizer = AutoTokenizer.from_pretrained(ppo_config.model_name)\n",
    "\n",
    "tokenizer.pad_token = tokenizer.eos_token\n",
    "generation_kwargs[\"pad_token_id\"] = tokenizer.eos_token_id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "7d8f326a-29fb-4c71-825c-f646d1a22baa",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "7d8f326a-29fb-4c71-825c-f646d1a22baa",
    "outputId": "584e561e-3750-429c-b759-9e96b93fc645"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sun Sep 15 17:24:13 2024       \n",
      "+---------------------------------------------------------------------------------------+\n",
      "| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2     |\n",
      "|-----------------------------------------+----------------------+----------------------+\n",
      "| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
      "| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |\n",
      "|                                         |                      |               MIG M. |\n",
      "|=========================================+======================+======================|\n",
      "|   0  Tesla T4                       Off | 00000000:00:04.0 Off |                    0 |\n",
      "| N/A   35C    P8               9W /  70W |      3MiB / 15360MiB |      0%      Default |\n",
      "|                                         |                      |                  N/A |\n",
      "+-----------------------------------------+----------------------+----------------------+\n",
      "                                                                                         \n",
      "+---------------------------------------------------------------------------------------+\n",
      "| Processes:                                                                            |\n",
      "|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |\n",
      "|        ID   ID                                                             Usage      |\n",
      "|=======================================================================================|\n",
      "|  No running processes found                                                           |\n",
      "+---------------------------------------------------------------------------------------+\n"
     ]
    }
   ],
   "source": [
    "!nvidia-smi"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7edff3ac-04d3-4834-bd78-ce71204490aa",
   "metadata": {
    "id": "7edff3ac-04d3-4834-bd78-ce71204490aa"
   },
   "source": [
    "## Prepare Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "fd06c08d-97a6-4211-9281-b9994b15eb79",
   "metadata": {
    "id": "fd06c08d-97a6-4211-9281-b9994b15eb79"
   },
   "outputs": [],
   "source": [
    "def prepare_dataset(\n",
    "    tokenizer, dataset_name=\"imdb\", input_min_text_length=2, input_max_text_length=8\n",
    "):\n",
    "    # load imdb with datasets\n",
    "    ds = load_dataset(dataset_name, split=\"train\")# ,cache_dir='/workspace'\n",
    "    ds = ds.rename_columns({\"text\": \"review\"})\n",
    "    ds = ds.filter(lambda x: len(x[\"review\"]) < 500, batched=False)\n",
    "\n",
    "    input_size = LengthSampler(input_min_text_length, input_max_text_length)\n",
    "\n",
    "    def tokenize(sample):\n",
    "        sample[\"input_ids\"] = tokenizer.encode(sample[\"review\"])[: input_size()]\n",
    "        sample[\"query\"] = tokenizer.decode(sample[\"input_ids\"])\n",
    "        return sample\n",
    "\n",
    "    ds = ds.map(tokenize, batched=False)\n",
    "    ds.set_format(type=\"torch\")\n",
    "    return ds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "a0de8e00-8083-4b8a-9632-53b29fdff156",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 305,
     "referenced_widgets": [
      "9bfe331c17c546a6ac58a1bb2fa874c7",
      "ff87ca71563b4cdb8d2cbf4c690c1922",
      "1f1f3151f6cf4b3da646f13253f5fbb1",
      "408b38405e21448fb939f2609e8c2cea",
      "ed4a5a90856340ae94db47241ac7e14a",
      "3e7a016083d44af6a1d8b047b803cdb0",
      "9e51a881928740c79ed471587b9f4e98",
      "477ee2c2aaa34dddab41b56142cd9659",
      "2c01a48af848479093a70016a13d24e8",
      "66eaee16e9ec490086dffd09369ccf60",
      "f260f5c57a2948ca82c3862639405f13",
      "318fd3adc242499885c56930f147566d",
      "8ca44fcac9be4aaab123128bd6743e68",
      "bd2c9ebf67a04a538f3bde44ce8107f3",
      "b75b711dc88f43f9b757697de1bdcf90",
      "8de59c241b574fcb83b8036ca44fef17",
      "059b7ac7e5e6451097d783e404cf7a7e",
      "df96c55c357b481fab626a687e44c6e0",
      "6b22ba5fc87440c1a19fb4ba3f0659ad",
      "0c73a3203dff4a949e7c15be100277c5",
      "5c3310bb65de4d178cc9a999a55185e1",
      "9dc38fd046024f95b711cc476cc138d8",
      "f57de9509e874cb38ef90cfc2e266a2f",
      "2bf0912777294d7384ac246fca984f94",
      "68fd2d482bdc40a38bd1d6bbeef7b8e0",
      "3758d7613cca4c6b9309e7e991cf03f5",
      "3bd73db8e99a42c2ad84ec04f32e8113",
      "3834ba5c3e1b49bc9e03a18c37a2d931",
      "3f6f527d09d54f7f9e38a465dbe04af6",
      "1dd60f1ebabe4d74994a51b688c73b80",
      "54c90b8fd8854de289642a284d145e29",
      "816feb637b724bca83ce7347d0b94547",
      "5f54bf184d3d4e8687543ecd27afc3cc",
      "d384ee5f00a6468489506ac470be90fa",
      "2bc64794e64d4cdfbd5222651a469857",
      "b65c9e0f00be4ff387b52094d11259ed",
      "493432db1fb84848964d14966627fc4b",
      "30844b55fec7479d82802a69b036eb1f",
      "9ebfe425886c46958067b0ce4aac1980",
      "2ce562645c3c43549851426f8c98bcd8",
      "02538b35512e40cc90c7da5c44d439f0",
      "4ad1469d6645401f9f8d52149fd2d03a",
      "46351724f1154e6892c089960a4b40a9",
      "300353225bed4b309dcc5e1df1d1afa0",
      "8563d658a4834e16a59636d8d6b1c512",
      "d8804046759d46c89f9a5cd2512d5b7a",
      "ed38c5091aad4446ab34b82f318531d7",
      "4341e013b9dd4c87ac30d7e74978834f",
      "0edf9f054c3c4e18976736d908a803f5",
      "61ee1b029cd348ecaa0cd6439ff904fc",
      "d297e0a54a8e4afe90d883ce17e22c05",
      "728d583ea43d45b9a002866a0a857871",
      "3f17fc1b5c2c4f1996151cbded670ae3",
      "1648bfefa8d146cdad215588de744fac",
      "2b8e1447938d4c2592da8096f70a0310",
      "82554e9776da462b9dd76d3393d67871",
      "cfcb16d2c2464e5cb52d55258ac22249",
      "bde58246b1bf4e2facd5bd24ec228783",
      "124fa02529c7427593b3bf71d4612646",
      "51425daae10f433987192f14faedde7b",
      "64bb37c8f5194853a8da7a149f0f6133",
      "c3f17dc960794f2385943498969978e2",
      "45ca8ac26b964d05bb5e42cdd31a012a",
      "fe307220f8124c888337c0f191186253",
      "54a77ee7423f4d649a371bc4a2909b60",
      "209090fc289c4eaeb2d69724e4141467",
      "16d37ae87d6046dcbb52e7acf3822e81",
      "62643405eecf4af2bad191b99bffd84c",
      "8a12f57eccaa4878a5f7d4f64234f02b",
      "680643b191104f38adcb2aef4cf7d5c1",
      "1204c9dc382e45e4a4fd62c5be9f4274",
      "6f34c93d9c1a45d5a9abe74c5fa4f365",
      "a821a5bb13444450b3bd785f20dfc421",
      "5115733ac8814b1b8eaf30d28dd62d15",
      "9d958d164c674abe9980ed3d2d5bdd9b",
      "5520bf0222074846a5ca67d326a68d0a",
      "a12bebd26bb946c59549a703b98c3878",
      "fe13e5e8b3d5440eb680e36f3ac6107d",
      "901639f4ef874a529c6a194a46fa5f32",
      "a1794db67fb74f7a9393b88c11ccf57b",
      "bd5a32bb27234df091bf15edd8859c00",
      "f4d8ccd596bd4b30bbbf2e5cf80c35f5",
      "0a70d9f2928841a5ab5a71a1f90beca6",
      "58d3dc01f23243448e73ae9dfaa32301",
      "53c80c8b1f9d4a1f9607be86275abe77",
      "d17b23a11a7c4e6db11c049bf3d4d69f",
      "96f0ee289dd54e89861f8977d883b201",
      "369fac0bbb324623acc8f2a9ad3c211e",
      "6d56b2617f0348eeb2d7d8a109646814",
      "017d20c252904df1bdb48bda9c8c1465",
      "274fa2d33c0247c1bb584751a64e28a7",
      "3b1c47003ad54aeaa3bfeae65a71ffc6",
      "ff50bca48f884f9b841a4881e1083b24",
      "030ab77c206e47c2884d8e3cfc811a36",
      "474c6a730a1f4222b99e3b4eede9c8cd",
      "a09a7a2ff8804931b7ad83ee773e0417",
      "f7037dc6dd074cc483e6482bea4581c1",
      "7b6bbb45ed2d4f77a848b6ba29d2f23b",
      "ca8c415219f4467b92c2d06ffa182c67"
     ]
    },
    "id": "a0de8e00-8083-4b8a-9632-53b29fdff156",
    "outputId": "3fd9f653-0de0-4e1a-f865-2d80b869428e"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "9bfe331c17c546a6ac58a1bb2fa874c7",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "README.md:   0%|          | 0.00/7.81k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "318fd3adc242499885c56930f147566d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "train-00000-of-00001.parquet:   0%|          | 0.00/21.0M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f57de9509e874cb38ef90cfc2e266a2f",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "test-00000-of-00001.parquet:   0%|          | 0.00/20.5M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d384ee5f00a6468489506ac470be90fa",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "unsupervised-00000-of-00001.parquet:   0%|          | 0.00/42.0M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8563d658a4834e16a59636d8d6b1c512",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Generating train split:   0%|          | 0/25000 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "82554e9776da462b9dd76d3393d67871",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Generating test split:   0%|          | 0/25000 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "16d37ae87d6046dcbb52e7acf3822e81",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Generating unsupervised split:   0%|          | 0/50000 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "fe13e5e8b3d5440eb680e36f3ac6107d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Filter:   0%|          | 0/25000 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "6d56b2617f0348eeb2d7d8a109646814",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/2410 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "dataset = prepare_dataset(tokenizer)\n",
    "\n",
    "\n",
    "def data_collator(data):\n",
    "    return dict((key, [d[key] for d in data]) for key in data[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20b99af4-7dcb-4bbb-a0a6-4ae052b00462",
   "metadata": {
    "id": "20b99af4-7dcb-4bbb-a0a6-4ae052b00462"
   },
   "source": [
    "## Setup PPO Trainer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "8ed7fe2c-202e-4f26-a0bf-f32fb28f23fb",
   "metadata": {
    "id": "8ed7fe2c-202e-4f26-a0bf-f32fb28f23fb"
   },
   "outputs": [],
   "source": [
    "ppo_trainer = PPOTrainer(ppo_config,\n",
    "                         model,\n",
    "                         ref_model,\n",
    "                         tokenizer,\n",
    "                         dataset,\n",
    "                         data_collator=data_collator,\n",
    "                         )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "fdc08b7b-37e5-4086-930f-5141d293580b",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "fdc08b7b-37e5-4086-930f-5141d293580b",
    "outputId": "94b6a65e-56e5-4339-d11e-e86a932cca32"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accelerator Device=0\n"
     ]
    }
   ],
   "source": [
    "if ppo_trainer.accelerator.num_processes == 1:\n",
    "    device = 0 if torch.cuda.is_available() else \"cpu\"\n",
    "else:\n",
    "    device = ppo_trainer.accelerator.device\n",
    "print(f\"Accelerator Device={device}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "59df473a-8ef4-4075-ad99-1faf93506a78",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "59df473a-8ef4-4075-ad99-1faf93506a78",
    "outputId": "724b2269-24ea-44fb-a5d5-2829dff9dd14"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "device(type='cuda')"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ppo_trainer.accelerator.device"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "66483a3e-e7b0-4acb-b8f8-d61fc046a796",
   "metadata": {
    "id": "66483a3e-e7b0-4acb-b8f8-d61fc046a796"
   },
   "source": [
    "### Setup Reward Model and Utils"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "272f3030-2639-436a-8e76-fc9ee8c32351",
   "metadata": {
    "id": "272f3030-2639-436a-8e76-fc9ee8c32351"
   },
   "source": [
    "### Reward Assignment\n",
    "\n",
    "The objective is to align our text generation model towards the alignment signal provided.\n",
    "To do so, we need to assign a corresponding reward to each output logit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "1715dab4-0733-430b-8575-dcff542dce9d",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 244,
     "referenced_widgets": [
      "534ac470d5974a57bdc7dd56177baea2",
      "7fb83f1ddbdd4223aadc1cec6960f4d9",
      "0f5b75071a944a7490e2aa111c4f79f9",
      "5834698eed4643a59c6092665a26e7c6",
      "2cce25a349ce479fbb0c8ee33c88eb6c",
      "040ec7d31dc24b269cc6593d98d26fe8",
      "74766eb042fc48c1b8fd5cab9143f5b1",
      "c84eb5f041614dc4bf577a45edc3b26d",
      "39b263537a664391af6612bd18b1043b",
      "dae1a036532f45b199f6cd3a703dfbaa",
      "db536d7ed52d4037b3aa89d8acb2c999",
      "9c880f7546e64072ab12908be8a9da94",
      "007d96ccd3ec436ab7b4e53589df8171",
      "096e494497254950b4c4eb2af68df123",
      "190f641967bb4bad9706a642ac7c2da2",
      "803e811bc7a14030a80366cdc714b9fa",
      "5c191aa20c87406f9b7e56c286d014f6",
      "1e5981a6ac3e48a78a8528c6de2fc7f6",
      "34025be36a4647bcba0a87e6f901e9b4",
      "d42211410ee64e7b9fff9c3653dea5b0",
      "91f8f77b67f34b6b8744b6018c3f2ed3",
      "e55a650c0f6f48fdb643ac0c3ecb203f",
      "408f2784f32d429099762d55c13b6fdd",
      "4b2a26e21c8548f29fe0cde2b1159dc4",
      "8c3269edbb0148259348b37cfd2ee694",
      "2caaf115ae0543a7b3cec697e73dfa48",
      "9561bbe794774920a2594c87cc2822b7",
      "af11a4a4d46d4f20825343a155cc5a50",
      "6b1a6e616e9e4664952e3594af71ae40",
      "e51622ab9b1a429799f6e823a41e58db",
      "24d1ae4c531f44549a69cbf13435b155",
      "601f9cc9a5794be48c4df19677e3ba27",
      "10a80c2039bd411581fe1dd895ebaac9",
      "09591085118748e888a175fedd09e749",
      "f2a33680c4ce4e8face4bbd17b2fec03",
      "d4f74fade7c94c2c88911394fe3f78dd",
      "8529ec8a1e594b84aca64f9ba6b5e5e0",
      "5907f9914df3490da8af4435fdd47471",
      "51c3e0e8584340a2ba1482adbd59cb06",
      "c217140ac4a64713bb1fe265d01782cd",
      "0c75945a099a4f6a850a2cdedfbc1d23",
      "c898ec71d4434765a94670abd972edb2",
      "dddb7ffab70e419ebe771557b84421d6",
      "0ba241098153433eb162fa4dcac0e6bc",
      "c47e110edc9943a99a2b02bdb1d1087d",
      "830bc949b24f49a49cc4fdd8de2db021",
      "11ce622e2580467d807fa8e7dd87e3b5",
      "49b4cd3787ef47399b90c14a627a67da",
      "9338547fd19542ac847d2e9ee95244a1",
      "579ccf522ee643e188f9c975d96b5e63",
      "aec19b7c46244887bb4be678639acb2c",
      "eac93a970ece49f1b43969cde67d6619",
      "9d5becf11dbe4b22a706147b799f2d51",
      "e76ba5f1a6ce49229bee1f51679d1efc",
      "fa2b4efbbfe0442580a85b0efae75d90",
      "59382ec30b664f0cb69bd1ee669a3412",
      "f4c605241d9843118aa091cb01063476",
      "d3fe49448a7f4d28bc0d7c3d60596989",
      "feb5600fce7445b6b6454a4aeb072e13",
      "9a731e2a431a4a039a7f9c7974c6dddc",
      "917afc28d0ae448c95cadf2d2cd9eeab",
      "cf06e4b58d0949cfac2c5025dc68d014",
      "490eddc9ab7048e8bfdc15ba8f5fa6a7",
      "d5736bbe814144878a30837e2efba596",
      "a32361ab80ce4528907e7969a762075d",
      "f2ca9a2ef8d34ffc9d5ef1923225f970"
     ]
    },
    "id": "1715dab4-0733-430b-8575-dcff542dce9d",
    "outputId": "0998ad7f-f493-4a78-e800-d4d135cb70b7"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "534ac470d5974a57bdc7dd56177baea2",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "tokenizer_config.json:   0%|          | 0.00/333 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "9c880f7546e64072ab12908be8a9da94",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "vocab.txt:   0%|          | 0.00/232k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "408f2784f32d429099762d55c13b6fdd",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "tokenizer.json:   0%|          | 0.00/466k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "09591085118748e888a175fedd09e749",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "special_tokens_map.json:   0%|          | 0.00/112 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "c47e110edc9943a99a2b02bdb1d1087d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "config.json:   0%|          | 0.00/735 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "59382ec30b664f0cb69bd1ee669a3412",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "pytorch_model.bin:   0%|          | 0.00/268M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "[{'label': 'NEGATIVE', 'score': 2.3350484371185303},\n",
       " {'label': 'POSITIVE', 'score': -2.726576328277588}]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## Get the Reward Model\n",
    "distilbert_tokenizer = AutoTokenizer.from_pretrained(\"lvwerra/distilbert-imdb\",eos_token='</s>')\n",
    "sentiment_pipe = pipeline(\"sentiment-analysis\", \"lvwerra/distilbert-imdb\", tokenizer=distilbert_tokenizer,device=device)\n",
    "\n",
    "# test out the pipeline\n",
    "text = \"this movie was really bad!!\"\n",
    "output = sentiment_pipe(text, **sentiment_pipe_kwargs)\n",
    "output"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6346baa6-53a9-4820-9933-f8d1d1962a5f",
   "metadata": {
    "id": "6346baa6-53a9-4820-9933-f8d1d1962a5f"
   },
   "source": [
    "## Time to Align using RLHF : PPO\n",
    "- Get a batch of queries and prepare training input\n",
    "- Get the query responses from the policy (model to be aligned)\n",
    "- Join query and responses and tokenize for reward based on sentiment analysis\n",
    "- Optimize policy with PPO using the (query, response, reward) triplet\n",
    "- Log all the training statistics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "529ca163-4f12-4a6b-9cc0-df3196dfe45d",
   "metadata": {
    "id": "529ca163-4f12-4a6b-9cc0-df3196dfe45d"
   },
   "outputs": [],
   "source": [
    "from tqdm.notebook import tqdm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "5bf2e88d-abb5-4527-b616-63e5e0fe406d",
   "metadata": {
    "id": "5bf2e88d-abb5-4527-b616-63e5e0fe406d"
   },
   "outputs": [],
   "source": [
    "output_min_length = 10\n",
    "output_max_length = 25\n",
    "num_steps = 5\n",
    "overall_rewards = list()\n",
    "output_length_sampler = LengthSampler(output_min_length, output_max_length)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "f5914ba7-bc24-48d7-aa2d-e55630c553c6",
   "metadata": {
    "id": "f5914ba7-bc24-48d7-aa2d-e55630c553c6"
   },
   "outputs": [],
   "source": [
    "generation_kwargs[\"pad_token_id\"]= tokenizer.eos_token_id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "TgGmqrwqlhQv",
   "metadata": {
    "id": "TgGmqrwqlhQv"
   },
   "outputs": [],
   "source": [
    "def get_rewards(output):\n",
    "  if output[0]['score']>output[1]['score']:\n",
    "    if output[0]['label'] == 'POSITIVE':\n",
    "      return torch.tensor(4*output[0]['score'])\n",
    "    else:\n",
    "      return torch.tensor(0.5*output[0]['score'])\n",
    "  elif output[1]['score']>output[0]['score']:\n",
    "    if output[1]['label'] == 'POSITIVE':\n",
    "      return torch.tensor(4*output[1]['score'])\n",
    "    else:\n",
    "      return torch.tensor(0.5*output[0]['score'])\n",
    "  return -1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "c096ba95-3f56-4a9d-83d9-fef3132ff814",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 434,
     "referenced_widgets": [
      "9d464f2073fb4b5e9f3e695314a4b082",
      "20ba4fcb1fe0441d9f8250904390a1bf",
      "f647c4105f9a4a6f9d749647eeae6526",
      "ff95bf3112384668bc376070c4cda552",
      "d814ab71fbc44031a6b132e2af8d7f0e",
      "f459c938c12542c3837b019c0770a760",
      "dcb3e035278646df81517bcbd63ad671",
      "585aa26fe56342aaa4b17d4d2876420e",
      "8e3691cfb96044e5bb17483a69b80b3a",
      "ab5e76fdf7d5439c8c0ee63a126923b2",
      "6831d41e645b4181b03c7f61d0205211"
     ]
    },
    "id": "c096ba95-3f56-4a9d-83d9-fef3132ff814",
    "outputId": "89d89172-31ad-4b70-b158-0194a10bdc61"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "9d464f2073fb4b5e9f3e695314a4b082",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "The attention mask is not set and cannot be inferred from input because pad token is same as eos token. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "objective/kl: 0.0\n",
      "ppo/returns/mean: 2.9586243629455566\n",
      "ppo/policy/advantages_mean: -4.513466222988427e-08\n",
      "---------------------------------------------------------------------------------------------------\n",
      "objective/kl: -0.0037075355648994446\n",
      "ppo/returns/mean: 2.8984031677246094\n",
      "ppo/policy/advantages_mean: 2.6163904820464268e-08\n",
      "---------------------------------------------------------------------------------------------------\n",
      "objective/kl: -0.005904155783355236\n",
      "ppo/returns/mean: 3.09124493598938\n",
      "ppo/policy/advantages_mean: -2.6478694792331225e-08\n",
      "---------------------------------------------------------------------------------------------------\n",
      "objective/kl: -0.027832351624965668\n",
      "ppo/returns/mean: 3.1628432273864746\n",
      "ppo/policy/advantages_mean: -4.605443137961629e-08\n",
      "---------------------------------------------------------------------------------------------------\n",
      "objective/kl: 0.13260036706924438\n",
      "ppo/returns/mean: 3.468047618865967\n",
      "ppo/policy/advantages_mean: 1.710036734436926e-08\n",
      "---------------------------------------------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "for epoch, batch in tqdm(enumerate(ppo_trainer.dataloader)):\n",
    "    if epoch >= num_steps:\n",
    "        break\n",
    "\n",
    "    query_tensors = batch[\"input_ids\"]\n",
    "\n",
    "    #### Get response from gpt2\n",
    "    response_tensors = []\n",
    "    for query in query_tensors:\n",
    "        gen_len = output_length_sampler()\n",
    "        generation_kwargs[\"max_new_tokens\"] = gen_len\n",
    "        response = ppo_trainer.generate(query, **generation_kwargs)\n",
    "        response_tensors.append(response.squeeze()[-gen_len:])\n",
    "    batch[\"response\"] = [tokenizer.decode(r.squeeze()) for r in response_tensors]\n",
    "\n",
    "    #### Compute sentiment score\n",
    "    texts = [q + r for q, r in zip(batch[\"query\"], batch[\"response\"])]\n",
    "    pipe_outputs = sentiment_pipe(texts, **sentiment_pipe_kwargs)\n",
    "    rewards = list()\n",
    "    for output in pipe_outputs:\n",
    "        rewards.append(get_rewards(output))\n",
    "\n",
    "    overall_rewards.append(rewards)\n",
    "    #PPO step\n",
    "    stats = ppo_trainer.step(query_tensors, response_tensors, rewards)\n",
    "\n",
    "    print(f'objective/kl: {stats[\"objective/kl\"]}')\n",
    "    print(f'ppo/returns/mean: {stats[\"ppo/returns/mean\"]}')\n",
    "    print(f'ppo/policy/advantages_mean: {stats[\"ppo/policy/advantages_mean\"]}')\n",
    "    print(\"-\".join(\"\" for x in range(100)))\n",
    "\n",
    "    ppo_trainer.log_stats(stats, batch, rewards)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "56fc6fff-6b88-46ce-8428-640944228626",
   "metadata": {
    "id": "56fc6fff-6b88-46ce-8428-640944228626"
   },
   "source": [
    "### Plot Reward Distribution"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "PSAHQdvxt_oy",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 430
    },
    "id": "PSAHQdvxt_oy",
    "outputId": "cf5c168c-8e02-4e53-d1be-9f0ed21fe5da"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAi8AAAGdCAYAAADaPpOnAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAsN0lEQVR4nO3de3BUZZ7/8U8u5MItXCK5YDRBGBgUEiQkG3RWZ+whspQ7qVUHKHeJ0dIqh7gw7QXiDIkWagCRyiCpRN31NjuMjFXi7LozsbDH4FgTLiZmEFEG3ShB7E6CkoYEEqa7f39YNL+WADkhyeknvF9VXXY//ZznfM8pTX98zi0iEAgEBAAAYIhIuwsAAACwgvACAACMQngBAABGIbwAAACjEF4AAIBRCC8AAMAohBcAAGAUwgsAADBKtN0F9Ae/36/Dhw9r1KhRioiIsLscAADQC4FAQMeOHVNqaqoiI3s/nzIkwsvhw4eVlpZmdxkAAKAPmpubdfnll/e6/5AIL6NGjZL07caPHj3a5moAAEBveL1epaWlBX/He2tIhJfTh4pGjx5NeAEAwDBWT/nghF0AAGAUwgsAADAK4QUAABhlSJzzAgCAqXw+n06dOmV3GQMmKipK0dHR/XorE8ILAAA2OX78uA4dOqRAIGB3KQNq+PDhSklJUUxMTL+MR3gBAMAGPp9Phw4d0vDhw3XZZZcNyZusBgIBdXd3q7W1VU1NTZoyZYqlm9GdC+EFAAAbnDp1SoFAQJdddpni4+PtLmfAxMfHa9iwYfriiy/U3d2tuLi4ix6TE3YBALDRUJxx+a7+mG0JGa9fRwMAABhghBcAAGAUznkBACCMvL3PM6jrc0xPGtT19QdmXgAAgGWVlZVKT09XXFyccnNztWvXrkFbN+EFAABYsmXLFjmdTpWVlamhoUGZmZnKz89XS0vLoKyf8AIAACzZsGGD7rnnHhUVFWn69Omqrq7W8OHD9cILLwzK+jnnZYDUNtfaXUKv3Zh2o90lAAAM0d3drfr6epWUlATbIiMj5XA4VFdXNyg1MPMCAAB6ra2tTT6fT0lJoSf6JiUlye12D0oNhBcAAGAUwgsAAOi1xMRERUVFyeMJvaTb4/EoOTl5UGogvAAAgF6LiYnR7Nmz5XK5gm1+v18ul0t5eXmDUgMn7AIAAEucTqcKCwuVnZ2tnJwcVVRUqKOjQ0VFRYOyfsILAABhxIQ73i5cuFCtra0qLS2V2+1WVlaWampqzjqJd6AQXgAAgGXFxcUqLi62Zd2c8wIAAIxCeAEAAEYhvAAAAKMQXgAAgFEILwAAwCiEFwAAYBTCCwAAMArhBQAAGIXwAgAAjMIddgEACCf7/zi465s63/Ii7777rp566inV19frq6++0tatW1VQUND/tZ0DMy8AAMCSjo4OZWZmqrKy0pb1M/MCAAAsmT9/vubPtz5j01+YeQEAAEYhvAAAAKMQXgAAgFEILwAAwCiEFwAAYJQ+hZfKykqlp6crLi5Oubm52rVr1zn7vv7668rOztaYMWM0YsQIZWVl6de//nVIn0AgoNLSUqWkpCg+Pl4Oh0MHDhzoS2kAAGCAHT9+XI2NjWpsbJQkNTU1qbGxUQcPHhyU9VsOL1u2bJHT6VRZWZkaGhqUmZmp/Px8tbS09Nh/3Lhx+sUvfqG6ujrt2bNHRUVFKioq0ltvvRXss27dOm3cuFHV1dXauXOnRowYofz8fJ08ebLvWwYAAAbE+++/r1mzZmnWrFmSJKfTqVmzZqm0tHRQ1h8RCAQCVhbIzc3VnDlztGnTJkmS3+9XWlqa7r//fq1cubJXY1x77bVasGCBVq9erUAgoNTUVD3wwAN68MEHJUnt7e1KSkrSSy+9pEWLFl1wPK/Xq4SEBLW3t2v06NFWNmfA1DbX2l1Cr92YdqPdJQDAJefkyZNqampSRkaG4uLi7C5nQJ1rW/v6+21p5qW7u1v19fVyOBxnBoiMlMPhUF1d3QWXDwQCcrlc2r9/v/7xH/9R0rdTTW63O2TMhIQE5ebmnnPMrq4ueb3ekBcAALg0WAovbW1t8vl8SkpKCmlPSkqS2+0+53Lt7e0aOXKkYmJitGDBAj3zzDP68Y9/LEnB5ayMWV5eroSEhOArLS3NymYAAACDDcrVRqNGjVJjY6N2796tJ554Qk6nU7W1tX0er6SkRO3t7cFXc3Nz/xULAADCmqVnGyUmJioqKkoejyek3ePxKDk5+ZzLRUZGavLkyZKkrKwsffzxxyovL9eNN94YXM7j8SglJSVkzKysrB7Hi42NVWxsrJXSAQDAEGFp5iUmJkazZ8+Wy+UKtvn9frlcLuXl5fV6HL/fr66uLklSRkaGkpOTQ8b0er3auXOnpTEBAMClwfJTpZ1OpwoLC5Wdna2cnBxVVFSoo6NDRUVFkqQlS5Zo4sSJKi8vl/Tt+SnZ2dm66qqr1NXVpT/84Q/69a9/raqqKklSRESEli9frscff1xTpkxRRkaGVq1apdTUVBUUFPTflgIAgCHBcnhZuHChWltbVVpaKrfbraysLNXU1ARPuD148KAiI89M6HR0dOhnP/uZDh06pPj4eE2bNk3/9V//pYULFwb7PPzww+ro6NC9996ro0eP6vrrr1dNTc2Qv3QMAABYZ/k+L+GI+7xcHO7zAgCDj/u89P332/LMC3qnsfmo3SX02o1caQ4AMAgPZgQAAEZh5gUAgDAy2KcdWD11oLy8XK+//ro++eQTxcfHa+7cuVq7dq2mTp06MAX2gJkXAADQa9u3b9fSpUu1Y8cObdu2TadOndK8efPU0dExaDUw8zIAjv3pHV32ycd2l9ErrVnft7sEAIBBampqQj6/9NJLmjBhgurr64PPLRxozLwAAIA+a29vlySNGzdu0NZJeAEAAH3i9/u1fPlyXXfddbrmmmsGbb0cNgIAAH2ydOlS7d27V++9996grpfwAgAALCsuLtabb76pd999V5dffvmgrpvwAgAAei0QCOj+++/X1q1bVVtbq4yMjEGvgfACAAB6benSpdq8ebN+//vfa9SoUXK73ZKkhIQExcfHD0oNhJcBMvzY53aX0Cujj5yS9sdKU+fbXQoAwABVVVWSpBtvvDGk/cUXX9Sdd945KDUQXgAACCPh/rDccHieM5dKAwAAoxBeAACAUQgvAADAKIQXAABgFMILAAAwCuEFAAAYhfACAACMQngBAABGIbwAAACjEF4AAIBReDwAAABh5Nif3hnU9Y360Q8t9a+qqlJVVZU+//xzSdLVV1+t0tJSzZ8/eM/IY+YFAAD02uWXX641a9aovr5e77//vn70ox/pJz/5iT766KNBq4GZFwAA0Gu33HJLyOcnnnhCVVVV2rFjh66++upBqYHwAgAA+sTn8+m1115TR0eH8vLyBm29hBcAAGDJhx9+qLy8PJ08eVIjR47U1q1bNX369EFbP+e8AAAAS6ZOnarGxkbt3LlT9913nwoLC7Vv375BWz8zLwAAwJKYmBhNnjxZkjR79mzt3r1bv/rVr/Tss88OyvqZeQEAABfF7/erq6tr0NbHzAsAAOi1kpISzZ8/X1dccYWOHTumzZs3q7a2Vm+99dag1UB4AQAAvdbS0qIlS5boq6++UkJCgmbOnKm33npLP/7xjwetBsILAABhxOodbwfbf/7nf9pdAue8AAAAsxBeAACAUQgvAADAKIQXAABgFMILAAAwCuEFAAAYhfACAACMQngBAABGIbwAAACj9OkOu5WVlXrqqafkdruVmZmpZ555Rjk5OT32ff755/XKK69o7969kr59+uSTTz4Z0v/OO+/Uyy+/HLJcfn6+ampq+lJev2va02ap/4nDAZ3sTB2gai4sbvhh29YNAMBAsxxetmzZIqfTqerqauXm5qqiokL5+fnav3+/JkyYcFb/2tpaLV68WHPnzlVcXJzWrl2refPm6aOPPtLEiROD/W6++Wa9+OKLwc+xsbF93CQAAMxl9X+YL1bGzMSLWn7NmjUqKSnRsmXLVFFR0T9FXYDlw0YbNmzQPffco6KiIk2fPl3V1dUaPny4XnjhhR77/+Y3v9HPfvYzZWVladq0afqP//gP+f1+uVyukH6xsbFKTk4OvsaOHdu3LQIAAINi9+7devbZZzVz5sxBXa+l8NLd3a36+no5HI4zA0RGyuFwqK6urldjdHZ26tSpUxo3blxIe21trSZMmKCpU6fqvvvu05EjR845RldXl7xeb8gLAAAMnuPHj+uOO+7Q888/P+gTDpbCS1tbm3w+n5KSkkLak5KS5Ha7ezXGihUrlJqaGhKAbr75Zr3yyityuVxau3attm/frvnz58vn8/U4Rnl5uRISEoKvtLQ0K5sBAAAu0tKlS7VgwYKQ3/PB0qcTdvtqzZo1evXVV1VbW6u4uLhg+6JFi4LvZ8yYoZkzZ+qqq65SbW2tbrrpprPGKSkpkdPpDH72er0EGAAABsmrr76qhoYG7d6925b1WwoviYmJioqKksfjCWn3eDxKTk4+77Lr16/XmjVr9Pbbb1/w2NikSZOUmJioTz/9tMfwEhsbywm9AADYoLm5WcuWLdO2bdtCJiIGk6XDRjExMZo9e3bIybanT77Ny8s753Lr1q3T6tWrVVNTo+zs7Auu59ChQzpy5IhSUlKslAcAAAZYfX29WlpadO211yo6OlrR0dHavn27Nm7cqOjo6HOe8tGfLB82cjqdKiwsVHZ2tnJyclRRUaGOjg4VFRVJkpYsWaKJEyeqvLxckrR27VqVlpZq8+bNSk9PD54bM3LkSI0cOVLHjx/XY489pltvvVXJycn67LPP9PDDD2vy5MnKz8/vx00FAAAX66abbtKHH34Y0lZUVKRp06ZpxYoVioqKGvAaLIeXhQsXqrW1VaWlpXK73crKylJNTU3wJN6DBw8qMvLMhE5VVZW6u7t12223hYxTVlamRx99VFFRUdqzZ49efvllHT16VKmpqZo3b55Wr17NoSEAAMLMqFGjdM0114S0jRgxQuPHjz+rfaD06YTd4uJiFRcX9/hdbW1tyOfPP//8vGPFx8frrbfe6ksZAADgEjSoVxsBAIDzu9g73trhuxMXA40HMwIAAKMQXgAAgFEILwAAwCiEFwAAYBTCCwAAMArhBQAAGwUCAbtLGHD9vY2EFwAAbHD6TrTd3d02VzLwOjs7JUnDhg3rl/G4zwsAADaIjo7W8OHD1draqmHDhoXcnX6oCAQC6uzsVEtLi8aMGdNvjw4gvAAAYIOIiAilpKSoqalJX3zxhd3lDKgxY8YoOTm538YjvAAAYJOYmBhNmTJlSB86GjZsWL8/rJHwAgCAjSIjIxUXF2d3GUYZegfYAADAkEZ4AQAARiG8AAAAoxBeAACAUQgvAADAKIQXAABgFMILAAAwCuEFAAAYhfACAACMQngBAABGIbwAAACjEF4AAIBRCC8AAMAohBcAAGAUwgsAADAK4QUAABiF8AIAAIxCeAEAAEYhvAAAAKMQXgAAgFEILwAAwCiEFwAAYBTCCwAAMArhBQAAGIXwAgAAjEJ4AQAARiG8AAAAoxBeAACAUQgvAADAKIQXAABgFMILAAAwSp/CS2VlpdLT0xUXF6fc3Fzt2rXrnH2ff/55/eAHP9DYsWM1duxYORyOs/oHAgGVlpYqJSVF8fHxcjgcOnDgQF9KAwAAQ5zl8LJlyxY5nU6VlZWpoaFBmZmZys/PV0tLS4/9a2trtXjxYr3zzjuqq6tTWlqa5s2bpy+//DLYZ926ddq4caOqq6u1c+dOjRgxQvn5+Tp58mTftwwAAAxJEYFAIGBlgdzcXM2ZM0ebNm2SJPn9fqWlpen+++/XypUrL7i8z+fT2LFjtWnTJi1ZskSBQECpqal64IEH9OCDD0qS2tvblZSUpJdeekmLFi264Jher1cJCQlqb2/X6NGjrWxOrzTtabPU/8SePfrrgT/3ex29FTf8cK/7fjN9ou763hxp6vwBrAgAgLP19ffb0sxLd3e36uvr5XA4zgwQGSmHw6G6urpejdHZ2alTp05p3LhxkqSmpia53e6QMRMSEpSbm3vOMbu6uuT1ekNeAADg0mApvLS1tcnn8ykpKSmkPSkpSW63u1djrFixQqmpqcGwcno5K2OWl5crISEh+EpLS7OyGQAAwGCDerXRmjVr9Oqrr2rr1q2Ki4vr8zglJSVqb28Pvpqbm/uxSgAAEM6irXROTExUVFSUPB5PSLvH41FycvJ5l12/fr3WrFmjt99+WzNnzgy2n17O4/EoJSUlZMysrKwex4qNjVVsbKyV0gEAwBBhaeYlJiZGs2fPlsvlCrb5/X65XC7l5eWdc7l169Zp9erVqqmpUXZ2dsh3GRkZSk5ODhnT6/Vq586d5x0TAABcmizNvEiS0+lUYWGhsrOzlZOTo4qKCnV0dKioqEiStGTJEk2cOFHl5eWSpLVr16q0tFSbN29Wenp68DyWkSNHauTIkYqIiNDy5cv1+OOPa8qUKcrIyNCqVauUmpqqgoKC/ttSAAAwJFgOLwsXLlRra6tKS0vldruVlZWlmpqa4Am3Bw8eVGTkmQmdqqoqdXd367bbbgsZp6ysTI8++qgk6eGHH1ZHR4fuvfdeHT16VNdff71qamou6rwYAAAwNFm+z0s44j4vobjPCwDABINynxcAAAC7EV4AAIBRCC8AAMAohBcAAGAUwgsAADAK4QUAABiF8AIAAIxCeAEAAEYhvAAAAKMQXgAAgFEILwAAwCiEFwAAYBTCCwAAMArhBQAAGIXwAgAAjEJ4AQAARiG8AAAAoxBeAACAUQgvAADAKIQXAABgFMILAAAwCuEFAAAYhfACAACMQngBAABGIbwAAACjEF4AAIBRCC8AAMAohBcAAGAUwgsAADAK4QUAABiF8AIAAIxCeAEAAEYhvAAAAKMQXgAAgFEILwAAwCiEFwAAYBTCCwAAMArhBQAAGIXwAgAAjEJ4AQAARiG8AAAAoxBeAACAUfoUXiorK5Wenq64uDjl5uZq165d5+z70Ucf6dZbb1V6eroiIiJUUVFxVp9HH31UERERIa9p06b1pTQAADDEWQ4vW7ZskdPpVFlZmRoaGpSZman8/Hy1tLT02L+zs1OTJk3SmjVrlJycfM5xr776an311VfB13vvvWe1NAAAcAmwHF42bNige+65R0VFRZo+fbqqq6s1fPhwvfDCCz32nzNnjp566iktWrRIsbGx5xw3OjpaycnJwVdiYqLV0gAAwCXAUnjp7u5WfX29HA7HmQEiI+VwOFRXV3dRhRw4cECpqamaNGmS7rjjDh08ePCixgMAAEOTpfDS1tYmn8+npKSkkPakpCS53e4+F5Gbm6uXXnpJNTU1qqqqUlNTk37wgx/o2LFjPfbv6uqS1+sNeQEAgEtDtN0FSNL8+fOD72fOnKnc3FxdeeWV+t3vfqe77777rP7l5eV67LHHBrNEAAAQJizNvCQmJioqKkoejyek3ePxnPdkXKvGjBmj733ve/r00097/L6kpETt7e3BV3Nzc7+tGwAAhDdL4SUmJkazZ8+Wy+UKtvn9frlcLuXl5fVbUcePH9dnn32mlJSUHr+PjY3V6NGjQ14AAODSYPmwkdPpVGFhobKzs5WTk6OKigp1dHSoqKhIkrRkyRJNnDhR5eXlkr49yXffvn3B919++aUaGxs1cuRITZ48WZL04IMP6pZbbtGVV16pw4cPq6ysTFFRUVq8eHF/bScAABgiLIeXhQsXqrW1VaWlpXK73crKylJNTU3wJN6DBw8qMvLMhM7hw4c1a9as4Of169dr/fr1uuGGG1RbWytJOnTokBYvXqwjR47osssu0/XXX68dO3bosssuu8jNAwAAQ01EIBAI2F3ExfJ6vUpISFB7e/uAHEJq2tNmqf+JPXv01wN/7vc6eitu+OFe9/1m+kTd9b050tT5F+4MAEA/6uvvN882AgAARiG8AAAAoxBeAACAUQgvAADAKIQXAABgFMILAAAwCuEFAAAYhfACAACMQngBAABGsfx4AAAAYF1tc63dJfTajWk32l3CeTHzAgAAjEJ4AQAARiG8AAAAoxBeAACAUQgvAADAKIQXAABgFMILAAAwCuEFAAAYhfACAACMQngBAABGIbwAAACjEF4AAIBReDAjAMAIx/70jt0l9MqoH/3Q7hKGPMILAAAD7O19Hu09etTuMnrtxjS7Kzg/DhsBAACjEF4AAIBRCC8AAMAohBcAAGAUwgsAADAK4QUAABiF8AIAAIxCeAEAAEYhvAAAAKNwh90h6GRnaq/7+jzj1BQVKXW1DWBFF5YxM9HW9QMAzMHMCwAAMArhBQAAGIXwAgAAjEJ4AQAARiG8AAAAo3C1EQBgaPiy3u4KvrX/5FlNiYePavSJAyFt3vEzB6uiIYfwAgAIe0172nTicOD8nb5OGJxiLiD+s0hlXOW3u4whjcNGAADAKIQXAABglD6Fl8rKSqWnpysuLk65ubnatWvXOft+9NFHuvXWW5Wenq6IiAhVVFRc9JgAAODSZTm8bNmyRU6nU2VlZWpoaFBmZqby8/PV0tLSY//Ozk5NmjRJa9asUXJycr+MCQAALl2WT9jdsGGD7rnnHhUVFUmSqqur9b//+7964YUXtHLlyrP6z5kzR3PmzJGkHr/vy5gAADS1doR8Hn78hE2VhAq0HVd7TLfdZQxplmZeuru7VV9fL4fDcWaAyEg5HA7V1dX1qYCBGBMAAAxdlmZe2tra5PP5lJSUFNKelJSkTz75pE8F9GXMrq4udXV1BT97vd4+rRsAAJjHyPu8lJeX67HHHrO7DAAAevT14Ziz2nzd40I+R3SG109wIO3vdpfQa5YOGyUmJioqKkoejyek3ePxnPNk3IEYs6SkRO3t7cFXc3Nzn9YNAADMYym8xMTEaPbs2XK5XME2v98vl8ulvLy8PhXQlzFjY2M1evTokBcAALg0WJ6zcjqdKiwsVHZ2tnJyclRRUaGOjo7glUJLlizRxIkTVV5eLunbE3L37dsXfP/ll1+qsbFRI0eO1OTJk3s1JgbOyOYjOtERK309zNY6jrVFXLDPqB/9cBAqAYCLE/HFlz22j/z7NyGfh8VHDUY559WekWZ3CX1iObwsXLhQra2tKi0tldvtVlZWlmpqaoIn3B48eFCRkWcmdA4fPqxZs2YFP69fv17r16/XDTfcoNra2l6NCQAAcFqfzhYqLi5WcXFxj9+dDiSnpaenKxC4wMO0LjAmAADAaeF1qjMA++3/o90VWDN1vt0VABhkPJgRAAAYhfACAACMQngBAABGIbwAAACjEF4AAIBRuNoI8nhPqNPfceGOA+hr34WfqXFyn0eO6dz7BwAudcy8AAAAoxBeAACAUQgvAADAKIQXAABgFMILAAAwCuEFAAAYhfACAACMwn1eAACwQeyJFrtL0PBjvuD7qCNfn/lif+zZncPoCe7MvAAAAKMQXgAAgFEILwAAwCiEFwAAYBTCCwAAMApXG8EYcbv/omPuMXaX0SujfvRDu0sAgCGL8AJJ0vDjTbauf7S+vnAnSYoc+e0/J84euGIAAGGNw0YAAMAohBcAAGAUwgsAADAK4QUAABiF8AIAAIxCeAEAAEbhUmnI/fdv7C5BR7u/6lW/DE0Z4Epgitpv9n37pjne3kJ64ca0G+0uARhSCC8Awk7TZxeeFD7WMfrbN80HBriai9f0YdeZD5eHxz2KMmYm2l0C0GccNgIAAEYhvAAAAKMQXgAAgFEILwAAwCiEFwAAYBSuNgKAAba34+CZD60x9hXy//miOaLHdtMu627tOhR8HxsGt33A4GDmBQAAGIWZFwC4BB37W6DH9qZv2ga5EsA6Zl4AAIBRCC8AAMAohBcAAGAUwgsAADAKJ+wCQD+L/qz13F+2htef3b9PTbe7BMCyPv1XVFlZqaeeekput1uZmZl65plnlJOTc87+r732mlatWqXPP/9cU6ZM0dq1a/VP//RPwe/vvPNOvfzyyyHL5Ofnq6ampi/lYQhrajsuSfL6j9pbyAWc3OeRJDmmJ0n7/2hzNbDT0ROnQj536aRNlfSsva3jOy0f2lJHb0R7vwj53NoVXvsSg8fyYaMtW7bI6XSqrKxMDQ0NyszMVH5+vlpaWnrs/5e//EWLFy/W3XffrQ8++EAFBQUqKCjQ3r17Q/rdfPPN+uqrr4Kv3/72t33bIgAAMKRZDi8bNmzQPffco6KiIk2fPl3V1dUaPny4XnjhhR77/+pXv9LNN9+shx56SN///ve1evVqXXvttdq0aVNIv9jYWCUnJwdfY8eO7dsWAQCAIc1SeOnu7lZ9fb0cDseZASIj5XA4VFdX1+MydXV1If2lbw8Jfbd/bW2tJkyYoKlTp+q+++7TkSNHzllHV1eXvF5vyAsAAFwaLIWXtrY2+Xw+JSUlhbQnJSXJ7Xb3uIzb7b5g/5tvvlmvvPKKXC6X1q5dq+3bt2v+/Pny+Xw9jlleXq6EhITgKy0tzcpmAAAAg4XFae+LFi0Kvp8xY4Zmzpypq666SrW1tbrpppvO6l9SUiKn0xn87PV6CTAAAFwiLIWXxMRERUVFyePxhLR7PB4lJyf3uExycrKl/pI0adIkJSYm6tNPP+0xvMTGxio2NtZK6QAMd+LAwZDP0SfOczkyei2hqTnkc/SROJsqAXrPUniJiYnR7Nmz5XK5VFBQIEny+/1yuVwqLi7ucZm8vDy5XC4tX7482LZt2zbl5eWdcz2HDh3SkSNHlJKSYqU8AEB/835ldwXnFPudy9Bx6bB8tZHT6dTzzz+vl19+WR9//LHuu+8+dXR0qKioSJK0ZMkSlZSUBPsvW7ZMNTU1evrpp/XJJ5/o0Ucf1fvvvx8MO8ePH9dDDz2kHTt26PPPP5fL5dJPfvITTZ48Wfn5+f20mQAAYKiwfM7LwoUL1draqtLSUrndbmVlZammpiZ4Uu7BgwcVGXkmE82dO1ebN2/WL3/5Sz3yyCOaMmWK3njjDV1zzTWSpKioKO3Zs0cvv/yyjh49qtTUVM2bN0+rV6/m0BAAADhLn07YLS4uPudhotra2rPabr/9dt1+++099o+Pj9dbb73VlzIAAMAliAczAgAAoxBeAACAUQgvAADAKGFxkzpgqHp7n0eJh4/aXUavNJw4YHcJQT7PuLPaRn597keGALi0EF6AS8DXh2Mu2MfXfXZgAIBwxGEjAABgFMILAAAwCuEFAAAYhfACAACMQngBAABGIbwAAACjcKk0AAyi2BMtdpdwXkdP2F0BcGHMvAAAAKMQXgAAgFEILwAAwCiEFwAAYBTCCwAAMArhBQAAGIVLpWGUz7q/kiR1dsTaXMn5tR6NCL4ffeKAjZV863xPjB7ZfOTbfw5WMQBwkQgvCAsnO1Mt9T8VGDMwhfTSsNFHbV0/AFzKOGwEAACMQngBAABGIbwAAACjEF4AAIBRCC8AAMAoXG0EI9n9ZN64iMPn/f7KP38+OIX00snOgN0lAEC/IbwAfWD10m4AQP/hsBEAADAK4QUAABiF8AIAAIxCeAEAAEYhvAAAAKMQXgAAgFEILwAAwCiEFwAAYBTCCwAAMArhBQAAGIXwAgAAjEJ4AQAARiG8AAAAoxBeAACAUQgvAADAKIQXAABglD6Fl8rKSqWnpysuLk65ubnatWvXefu/9tprmjZtmuLi4jRjxgz94Q9/CPk+EAiotLRUKSkpio+Pl8Ph0IEDB/pSGgAAGOIsh5ctW7bI6XSqrKxMDQ0NyszMVH5+vlpaWnrs/5e//EWLFy/W3XffrQ8++EAFBQUqKCjQ3r17g33WrVunjRs3qrq6Wjt37tSIESOUn5+vkydP9n3LAADAkBQRCAQCVhbIzc3VnDlztGnTJkmS3+9XWlqa7r//fq1cufKs/gsXLlRHR4fefPPNYNs//MM/KCsrS9XV1QoEAkpNTdUDDzygBx98UJLU3t6upKQkvfTSS1q0aNEFa/J6vUpISFB7e7tGjx5tZXN6pWlPm6X+J/bs0V8P/Lnf6wAAoD8dTxsffB+V9HXw/V3fm3N256nz+339ff39jrayku7ubtXX16ukpCTYFhkZKYfDobq6uh6Xqaurk9PpDGnLz8/XG2+8IUlqamqS2+2Ww+EIfp+QkKDc3FzV1dX1GF66urrU1dUV/Nze3i7p250wEI4dP2ap/8kTHersYtYIABDeTpw4EXwf1Xnmd8t7vPPszgPwG3v6d9viPIq18NLW1iafz6ekpKSQ9qSkJH3yySc9LuN2u3vs73a7g9+fbjtXn+8qLy/XY489dlZ7Wlpa7zYEAACcU/Egr+/YsWNKSEjodX9L4SVclJSUhMzm+P1+ff311xo/frwiIiJsrOwMr9ertLQ0NTc3D8ihrKGG/dV77Ctr2F+9x76yhv3Ve+faV4FAQMeOHVNqaqql8SyFl8TEREVFRcnj8YS0ezweJScn97hMcnLyefuf/qfH41FKSkpIn6ysrB7HjI2NVWxsbEjbmDFjrGzKoBk9ejT/UlvA/uo99pU17K/eY19Zw/7qvZ72lZUZl9MsXW0UExOj2bNny+VyBdv8fr9cLpfy8vJ6XCYvLy+kvyRt27Yt2D8jI0PJyckhfbxer3bu3HnOMQEAwKXL8mEjp9OpwsJCZWdnKycnRxUVFero6FBRUZEkacmSJZo4caLKy8slScuWLdMNN9ygp59+WgsWLNCrr76q999/X88995wkKSIiQsuXL9fjjz+uKVOmKCMjQ6tWrVJqaqoKCgr6b0sBAMCQYDm8LFy4UK2trSotLZXb7VZWVpZqamqCJ9wePHhQkZFnJnTmzp2rzZs365e//KUeeeQRTZkyRW+88YauueaaYJ+HH35YHR0duvfee3X06FFdf/31qqmpUVxcXD9soj1iY2NVVlZ21uEt9Iz91XvsK2vYX73HvrKG/dV7/b2vLN/nBQAAwE482wgAABiF8AIAAIxCeAEAAEYhvAAAAKMQXgZIZWWl0tPTFRcXp9zcXO3atcvuksJOeXm55syZo1GjRmnChAkqKCjQ/v377S7LGGvWrAneagBn+/LLL/Wv//qvGj9+vOLj4zVjxgy9//77dpcVlnw+n1atWqWMjAzFx8frqquu0urVqy0/b2aoevfdd3XLLbcoNTVVERERwWfznRYIBFRaWqqUlBTFx8fL4XDowIED9hRrs/Ptq1OnTmnFihWaMWOGRowYodTUVC1ZskSHDx+2vB7CywDYsmWLnE6nysrK1NDQoMzMTOXn56ulpcXu0sLK9u3btXTpUu3YsUPbtm3TqVOnNG/ePHV0dNhdWtjbvXu3nn32Wc2cOdPuUsLSN998o+uuu07Dhg3TH//4R+3bt09PP/20xo4da3dpYWnt2rWqqqrSpk2b9PHHH2vt2rVat26dnnnmGbtLCwsdHR3KzMxUZWVlj9+vW7dOGzduVHV1tXbu3KkRI0YoPz9fJ09eeg/oPd++6uzsVENDg1atWqWGhga9/vrr2r9/v/75n//Z+ooC6Hc5OTmBpUuXBj/7fL5AampqoLy83Maqwl9LS0tAUmD79u12lxLWjh07FpgyZUpg27ZtgRtuuCGwbNkyu0sKOytWrAhcf/31dpdhjAULFgTuuuuukLZ/+Zd/Cdxxxx02VRS+JAW2bt0a/Oz3+wPJycmBp556Kth29OjRQGxsbOC3v/2tDRWGj+/uq57s2rUrICnwxRdfWBqbmZd+1t3drfr6ejkcjmBbZGSkHA6H6urqbKws/LW3t0uSxo0bZ3Ml4W3p0qVasGBByL9jCPXf//3fys7O1u23364JEyZo1qxZev755+0uK2zNnTtXLpdLf/vb3yRJf/3rX/Xee+9p/vz5NlcW/pqamuR2u0P+e0xISFBubi5/83uhvb1dERERlp9PaORTpcNZW1ubfD5f8I7DpyUlJemTTz6xqarw5/f7tXz5cl133XUhd19GqFdffVUNDQ3avXu33aWEtf/7v/9TVVWVnE6nHnnkEe3evVv//u//rpiYGBUWFtpdXthZuXKlvF6vpk2bpqioKPl8Pj3xxBO644477C4t7Lndbknq8W/+6e/Qs5MnT2rFihVavHix5QdbEl4QFpYuXaq9e/fqvffes7uUsNXc3Kxly5Zp27ZtRj86YzD4/X5lZ2frySeflCTNmjVLe/fuVXV1NeGlB7/73e/0m9/8Rps3b9bVV1+txsZGLV++XKmpqewvDIhTp07ppz/9qQKBgKqqqiwvz2GjfpaYmKioqCh5PJ6Qdo/Ho+TkZJuqCm/FxcV688039c477+jyyy+3u5ywVV9fr5aWFl177bWKjo5WdHS0tm/fro0bNyo6Olo+n8/uEsNGSkqKpk+fHtL2/e9/XwcPHrSpovD20EMPaeXKlVq0aJFmzJihf/u3f9PPf/7z4AN2cW6n/67zN7/3TgeXL774Qtu2bbM86yIRXvpdTEyMZs+eLZfLFWzz+/1yuVzKy8uzsbLwEwgEVFxcrK1bt+pPf/qTMjIy7C4prN1000368MMP1djYGHxlZ2frjjvuUGNjo6KiouwuMWxcd911Z112/7e//U1XXnmlTRWFt87OzpAH6kpSVFSU/H6/TRWZIyMjQ8nJySF/871er3bu3Mnf/B6cDi4HDhzQ22+/rfHjx/dpHA4bDQCn06nCwkJlZ2crJydHFRUV6ujoUFFRkd2lhZWlS5dq8+bN+v3vf69Ro0YFjw8nJCQoPj7e5urCz6hRo846H2jEiBEaP3485wl9x89//nPNnTtXTz75pH76059q165deu655/Tcc8/ZXVpYuuWWW/TEE0/oiiuu0NVXX60PPvhAGzZs0F133WV3aWHh+PHj+vTTT4Ofm5qa1NjYqHHjxumKK67Q8uXL9fjjj2vKlCnKyMjQqlWrlJqaqoKCAvuKtsn59lVKSopuu+02NTQ06M0335TP5wv+3R83bpxiYmJ6v6I+XwOF83rmmWcCV1xxRSAmJiaQk5MT2LFjh90lhR1JPb5efPFFu0szBpdKn9v//M//BK655ppAbGxsYNq0aYHnnnvO7pLCltfrDSxbtixwxRVXBOLi4gKTJk0K/OIXvwh0dXXZXVpYeOedd3r8W1VYWBgIBL69XHrVqlWBpKSkQGxsbOCmm24K7N+/396ibXK+fdXU1HTOv/vvvPOOpfVEBALcQhEAAJiDc14AAIBRCC8AAMAohBcAAGAUwgsAADAK4QUAABiF8AIAAIxCeAEAAEYhvAAAAKMQXgAAgFEILwAAwCiEFwAAYBTCCwAAMMr/A2lMba/o9niVAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "for idx,epoch_rewards in enumerate(overall_rewards):\n",
    "    plt.hist([i.item() for i in epoch_rewards], density=True, alpha=0.3)\n",
    "plt.legend(range(10))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3dad9f84-64ad-4ac7-9367-afd3974f1a2c",
   "metadata": {
    "id": "3dad9f84-64ad-4ac7-9367-afd3974f1a2c"
   },
   "source": [
    "### Compare Rewards"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "8eb8a23d-1c3f-4c74-b3e9-c372b1c5cb54",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 551
    },
    "id": "8eb8a23d-1c3f-4c74-b3e9-c372b1c5cb54",
    "outputId": "175aed96-8b6b-46bb-d57c-12185a0bebf5"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.google.colaboratory.intrinsic+json": {
       "summary": "{\n  \"name\": \"df_results\",\n  \"rows\": 16,\n  \"fields\": [\n    {\n      \"column\": \"query\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 16,\n        \"samples\": [\n          \"As usual,\",\n          \"This is a bit of a\",\n          \"I thought\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"response (before)\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 16,\n        \"samples\": [\n          \" my story. I do not care if the plot is witty or easy to understand, Zero Dawn's plot\",\n          \" considering the money and the clothes coming out of learning to fight. The second time Zack is in the movie\",\n          \" was taking the opportunity to:-) if and only how-I ended up this one. Time spent entertainingadventure\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"response (after)\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 16,\n        \"samples\": [\n          \", as usual. The only redeeming aspect of this DVD is a horrible ending that is both somewhat decent\",\n          \". The plot is VERY simple, perhaps even worse. The story and locations are so light, the story\",\n          \" it would be somewhat redundant. But this was not the movie. It was very bad for many years. It was\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"rewards (before)\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 4.418084160754302,\n        \"min\": -0.013747422024607658,\n        \"max\": 10.845561981201172,\n        \"num_unique_values\": 16,\n        \"samples\": [\n          0.8415138125419617,\n          3.745473623275757,\n          0.3026203215122223\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"rewards (after)\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 3.6562010183750884,\n        \"min\": 0.0026395232416689396,\n        \"max\": 11.049273490905762,\n        \"num_unique_values\": 16,\n        \"samples\": [\n          1.0808472633361816,\n          2.730339527130127,\n          1.1052809953689575\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}",
       "type": "dataframe",
       "variable_name": "df_results"
      },
      "text/html": [
       "\n",
       "  <div id=\"df-7051dbab-8b0e-4a0c-8280-9d513311476d\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>query</th>\n",
       "      <th>response (before)</th>\n",
       "      <th>response (after)</th>\n",
       "      <th>rewards (before)</th>\n",
       "      <th>rewards (after)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>As usual,</td>\n",
       "      <td>my story. I do not care if the plot is witty ...</td>\n",
       "      <td>, as usual. The only redeeming aspect of this ...</td>\n",
       "      <td>0.841514</td>\n",
       "      <td>1.080847</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>This is a bit of a</td>\n",
       "      <td>considering the money and the clothes coming ...</td>\n",
       "      <td>. The plot is VERY simple, perhaps even worse....</td>\n",
       "      <td>3.745474</td>\n",
       "      <td>2.730340</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Wonderful film that mixes documentary and</td>\n",
       "      <td>cinematic style with punk rock smiles. This f...</td>\n",
       "      <td>old-fashioned, is unlike anything else. I hop...</td>\n",
       "      <td>10.201181</td>\n",
       "      <td>11.049273</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Powers Boothe turns</td>\n",
       "      <td>stick in the frame! Very smooth, smooth, smoo...</td>\n",
       "      <td>in film, Angelina Goode is a gorgeous woman w...</td>\n",
       "      <td>10.827898</td>\n",
       "      <td>5.603095</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>This is</td>\n",
       "      <td>to look great.John Floyd is an outstanding ex...</td>\n",
       "      <td>is really creepy. No sometimes sad, but this ...</td>\n",
       "      <td>9.184366</td>\n",
       "      <td>0.002640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>I thought</td>\n",
       "      <td>was taking the opportunity to:-) if and only ...</td>\n",
       "      <td>it would be somewhat redundant. But this was ...</td>\n",
       "      <td>0.302620</td>\n",
       "      <td>1.105281</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>\"Tulip\" is on</td>\n",
       "      <td>with children under 18, but dwindling over th...</td>\n",
       "      <td>-loud....\"EXTREME\" is on call. It's like a mov...</td>\n",
       "      <td>0.104786</td>\n",
       "      <td>0.152721</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>The first</td>\n",
       "      <td>anything resembling Marvel in the United Stat...</td>\n",
       "      <td>issinfo, I Was almost bummed by the introducti...</td>\n",
       "      <td>1.095995</td>\n",
       "      <td>0.366820</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Firstly let me say that</td>\n",
       "      <td>and it's incredibly well done - I have been s...</td>\n",
       "      <td>ing out, AmyKlosenbert does her heart strength...</td>\n",
       "      <td>8.721676</td>\n",
       "      <td>5.614421</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>This is simply the</td>\n",
       "      <td>in America.One of the films borrows a</td>\n",
       "      <td>do what you wish but you can seldom find the</td>\n",
       "      <td>0.471817</td>\n",
       "      <td>4.357355</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>An annoying experience. Impro</td>\n",
       "      <td>. May be abused to a point of performance. Som...</td>\n",
       "      <td>John Waters and Madloles).grossingatingMovie ...</td>\n",
       "      <td>0.957047</td>\n",
       "      <td>0.415135</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>The only reason to see this movie</td>\n",
       "      <td>. Unfortunately, most of the crimes were commi...</td>\n",
       "      <td>the idiot, the father &amp; son, and the kidsuste...</td>\n",
       "      <td>1.077971</td>\n",
       "      <td>1.211308</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>This movie is pretty cheesy</td>\n",
       "      <td>it. For great breasts, the characters don't a...</td>\n",
       "      <td>all time.I liked the script and it 2 in pairs...</td>\n",
       "      <td>-0.013747</td>\n",
       "      <td>8.965014</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>It was nice to see</td>\n",
       "      <td>Looks. I need someone for this movie. It was ...</td>\n",
       "      <td>it wasnice. I would recommend this movie to a...</td>\n",
       "      <td>5.511101</td>\n",
       "      <td>9.160538</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>102 DALMATIANS [</td>\n",
       "      <td>2x100). Camera 3 with focus lens, film effect</td>\n",
       "      <td>antos on the web at www dubbed 'Xov</td>\n",
       "      <td>0.034242</td>\n",
       "      <td>0.112770</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Busy Phillips put in</td>\n",
       "      <td>funny bits were worth watching and reaching o...</td>\n",
       "      <td>wrote and directed one for My husband Landie....</td>\n",
       "      <td>10.845562</td>\n",
       "      <td>2.850229</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-7051dbab-8b0e-4a0c-8280-9d513311476d')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-7051dbab-8b0e-4a0c-8280-9d513311476d button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-7051dbab-8b0e-4a0c-8280-9d513311476d');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-aacab8d6-a890-4d79-b637-41493a56b3e0\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-aacab8d6-a890-4d79-b637-41493a56b3e0')\"\n",
       "            title=\"Suggest charts\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-aacab8d6-a890-4d79-b637-41493a56b3e0 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "\n",
       "  <div id=\"id_d9072d09-b5d3-46a1-85ab-035309aac8c1\">\n",
       "    <style>\n",
       "      .colab-df-generate {\n",
       "        background-color: #E8F0FE;\n",
       "        border: none;\n",
       "        border-radius: 50%;\n",
       "        cursor: pointer;\n",
       "        display: none;\n",
       "        fill: #1967D2;\n",
       "        height: 32px;\n",
       "        padding: 0 0 0 0;\n",
       "        width: 32px;\n",
       "      }\n",
       "\n",
       "      .colab-df-generate:hover {\n",
       "        background-color: #E2EBFA;\n",
       "        box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "        fill: #174EA6;\n",
       "      }\n",
       "\n",
       "      [theme=dark] .colab-df-generate {\n",
       "        background-color: #3B4455;\n",
       "        fill: #D2E3FC;\n",
       "      }\n",
       "\n",
       "      [theme=dark] .colab-df-generate:hover {\n",
       "        background-color: #434B5C;\n",
       "        box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "        filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "        fill: #FFFFFF;\n",
       "      }\n",
       "    </style>\n",
       "    <button class=\"colab-df-generate\" onclick=\"generateWithVariable('df_results')\"\n",
       "            title=\"Generate code using this dataframe.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "       width=\"24px\">\n",
       "    <path d=\"M7,19H8.4L18.45,9,17,7.55,7,17.6ZM5,21V16.75L18.45,3.32a2,2,0,0,1,2.83,0l1.4,1.43a1.91,1.91,0,0,1,.58,1.4,1.91,1.91,0,0,1-.58,1.4L9.25,21ZM18.45,9,17,7.55Zm-12,3A5.31,5.31,0,0,0,4.9,8.1,5.31,5.31,0,0,0,1,6.5,5.31,5.31,0,0,0,4.9,4.9,5.31,5.31,0,0,0,6.5,1,5.31,5.31,0,0,0,8.1,4.9,5.31,5.31,0,0,0,12,6.5,5.46,5.46,0,0,0,6.5,12Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "    <script>\n",
       "      (() => {\n",
       "      const buttonEl =\n",
       "        document.querySelector('#id_d9072d09-b5d3-46a1-85ab-035309aac8c1 button.colab-df-generate');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      buttonEl.onclick = () => {\n",
       "        google.colab.notebook.generateWithVariable('df_results');\n",
       "      }\n",
       "      })();\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                        query  \\\n",
       "0                                   As usual,   \n",
       "1                          This is a bit of a   \n",
       "2   Wonderful film that mixes documentary and   \n",
       "3                         Powers Boothe turns   \n",
       "4                                     This is   \n",
       "5                                   I thought   \n",
       "6                               \"Tulip\" is on   \n",
       "7                                   The first   \n",
       "8                     Firstly let me say that   \n",
       "9                          This is simply the   \n",
       "10              An annoying experience. Impro   \n",
       "11          The only reason to see this movie   \n",
       "12                This movie is pretty cheesy   \n",
       "13                         It was nice to see   \n",
       "14                           102 DALMATIANS [   \n",
       "15                       Busy Phillips put in   \n",
       "\n",
       "                                    response (before)  \\\n",
       "0    my story. I do not care if the plot is witty ...   \n",
       "1    considering the money and the clothes coming ...   \n",
       "2    cinematic style with punk rock smiles. This f...   \n",
       "3    stick in the frame! Very smooth, smooth, smoo...   \n",
       "4    to look great.John Floyd is an outstanding ex...   \n",
       "5    was taking the opportunity to:-) if and only ...   \n",
       "6    with children under 18, but dwindling over th...   \n",
       "7    anything resembling Marvel in the United Stat...   \n",
       "8    and it's incredibly well done - I have been s...   \n",
       "9               in America.One of the films borrows a   \n",
       "10  . May be abused to a point of performance. Som...   \n",
       "11  . Unfortunately, most of the crimes were commi...   \n",
       "12   it. For great breasts, the characters don't a...   \n",
       "13   Looks. I need someone for this movie. It was ...   \n",
       "14      2x100). Camera 3 with focus lens, film effect   \n",
       "15   funny bits were worth watching and reaching o...   \n",
       "\n",
       "                                     response (after)  rewards (before)  \\\n",
       "0   , as usual. The only redeeming aspect of this ...          0.841514   \n",
       "1   . The plot is VERY simple, perhaps even worse....          3.745474   \n",
       "2    old-fashioned, is unlike anything else. I hop...         10.201181   \n",
       "3    in film, Angelina Goode is a gorgeous woman w...         10.827898   \n",
       "4    is really creepy. No sometimes sad, but this ...          9.184366   \n",
       "5    it would be somewhat redundant. But this was ...          0.302620   \n",
       "6   -loud....\"EXTREME\" is on call. It's like a mov...          0.104786   \n",
       "7   issinfo, I Was almost bummed by the introducti...          1.095995   \n",
       "8   ing out, AmyKlosenbert does her heart strength...          8.721676   \n",
       "9        do what you wish but you can seldom find the          0.471817   \n",
       "10   John Waters and Madloles).grossingatingMovie ...          0.957047   \n",
       "11   the idiot, the father & son, and the kidsuste...          1.077971   \n",
       "12   all time.I liked the script and it 2 in pairs...         -0.013747   \n",
       "13   it wasnice. I would recommend this movie to a...          5.511101   \n",
       "14                antos on the web at www dubbed 'Xov          0.034242   \n",
       "15   wrote and directed one for My husband Landie....         10.845562   \n",
       "\n",
       "    rewards (after)  \n",
       "0          1.080847  \n",
       "1          2.730340  \n",
       "2         11.049273  \n",
       "3          5.603095  \n",
       "4          0.002640  \n",
       "5          1.105281  \n",
       "6          0.152721  \n",
       "7          0.366820  \n",
       "8          5.614421  \n",
       "9          4.357355  \n",
       "10         0.415135  \n",
       "11         1.211308  \n",
       "12         8.965014  \n",
       "13         9.160538  \n",
       "14         0.112770  \n",
       "15         2.850229  "
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bs = 16\n",
    "game_data = dict()\n",
    "dataset.set_format(\"pandas\")\n",
    "df_batch = dataset[:].sample(bs)\n",
    "game_data[\"query\"] = df_batch[\"query\"].tolist()\n",
    "query_tensors = df_batch[\"input_ids\"].tolist()\n",
    "\n",
    "response_tensors_ref, response_tensors = [], []\n",
    "\n",
    "#### get response from phi2 and phi2_ref\n",
    "for i in range(bs):\n",
    "    gen_len = output_length_sampler()\n",
    "    output = ref_model.generate(\n",
    "        torch.tensor(query_tensors[i]).unsqueeze(dim=0).to(device),\n",
    "        # max_new_tokens=gen_len,\n",
    "        **generation_kwargs\n",
    "    ).squeeze()[-gen_len:]\n",
    "    response_tensors_ref.append(output)\n",
    "    output = model.generate(\n",
    "        input_ids=torch.tensor(query_tensors[i]).unsqueeze(dim=0).to(device),\n",
    "        # max_new_tokens=gen_len,\n",
    "        **generation_kwargs\n",
    "    ).squeeze()[-gen_len:]\n",
    "    response_tensors.append(output)\n",
    "\n",
    "#### decode responses\n",
    "game_data[\"response (before)\"] = [\n",
    "    tokenizer.decode(response_tensors_ref[i]) for i in range(bs)\n",
    "]\n",
    "game_data[\"response (after)\"] = [\n",
    "    tokenizer.decode(response_tensors[i]) for i in range(bs)\n",
    "]\n",
    "\n",
    "#### sentiment analysis of query/response pairs before/after\n",
    "texts = [q + r for q, r in zip(game_data[\"query\"], game_data[\"response (before)\"])]\n",
    "# game_data[\"rewards (before)\"] = [\n",
    "#     output[1][\"score\"] for output in sentiment_pipe(texts, **sentiment_pipe_kwargs)\n",
    "# ]\n",
    "game_data[\"rewards (before)\"]=list()\n",
    "for output in sentiment_pipe(texts, **sentiment_pipe_kwargs):\n",
    "        if output[0]['score']>output[1]['score']:\n",
    "            if output[0]['label'] == 'POSITIVE':\n",
    "                game_data[\"rewards (before)\"].append(4*output[0]['score'])\n",
    "            else:\n",
    "                game_data[\"rewards (before)\"].append(0.5*output[0]['score'])\n",
    "        elif output[1]['score']>output[0]['score']:\n",
    "            if output[1]['label'] == 'POSITIVE':\n",
    "                game_data[\"rewards (before)\"].append(4*output[1]['score'])\n",
    "            else:\n",
    "                game_data[\"rewards (before)\"].append(0.5*output[0]['score'])\n",
    "\n",
    "texts = [q + r for q, r in zip(game_data[\"query\"], game_data[\"response (after)\"])]\n",
    "\n",
    "game_data[\"rewards (after)\"]=list()\n",
    "for output in sentiment_pipe(texts, **sentiment_pipe_kwargs):\n",
    "        if output[0]['score']>output[1]['score']:\n",
    "            if output[0]['label'] == 'POSITIVE':\n",
    "                game_data[\"rewards (after)\"].append(4*output[0]['score'])\n",
    "            else:\n",
    "                game_data[\"rewards (after)\"].append(0.5*output[0]['score'])\n",
    "        elif output[1]['score']>output[0]['score']:\n",
    "            if output[1]['label'] == 'POSITIVE':\n",
    "                game_data[\"rewards (after)\"].append(4*output[1]['score'])\n",
    "            else:\n",
    "                game_data[\"rewards (after)\"].append(0.5*output[0]['score'])\n",
    "\n",
    "\n",
    "# store results in a dataframe\n",
    "df_results = pd.DataFrame(game_data)\n",
    "df_results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "xk7cLZUbu1Fo",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 452
    },
    "id": "xk7cLZUbu1Fo",
    "outputId": "07673d64-e634-4dd7-cd45-48c7ff7413d3"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAioAAAGzCAYAAAABsTylAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAnXUlEQVR4nO3deXRU9f3/8deQkEkMWSABgQZCCmENYAFRZFEBQUFk6YECoUbA9utpaBNTF/L1IHIQA1KioghKMSiVTQVbsUjZhFBA9lUFVCKLYCJiQiIMmLm/P/wxX4cEgWHi/UzyfJwzf8xnbu68c08kT+/cmTgsy7IEAABgoGp2DwAAAHA5hAoAADAWoQIAAIxFqAAAAGMRKgAAwFiECgAAMBahAgAAjEWoAAAAYxEqAADAWIQKANvMnTtXDodDeXl5ftmf2+1WUlKSJk2a5NPXHzp0SL169VJUVJQcDofeffddv8x1OadOnVJ4eLj+/e9/V+jzAIGMUAFQaSxYsEBHjx7VmDFjfPr6lJQU7d27V5MmTdK8efPUoUMHzZ8/X88//7x/B/3/YmJi9OCDD2rcuHEVsn+gMiBUAFQaU6dO1dChQxUVFXXNX3v27Flt2rRJo0eP1pgxYzRixAjFxcVVaKhI0kMPPaQdO3ZozZo1FfYcQCAjVIAAVlJSYvcIP8vtduvcuXO/yHPt3LlTu3fv1pAhQ3z6+oKCAklSdHS0H6cq30+PS4sWLZSUlKS5c+dW+PMCgYhQAQLEU089JYfDoY8//ljDhw9XzZo11aVLF8/j//jHP9S+fXuFhYWpVq1aGjp0qI4ePep5fPr06QoKCtJ3333nWZs2bZocDocyMjI8a6WlpYqIiNDjjz/uWfvb3/6m2267TTExMQoLC1P79u319ttvl5nR4XBozJgxevPNN9WqVSs5nU598MEHkqT9+/ere/fuCgsLU1xcnJ5++mm53e4y+9i2bZt69+6t2NhYhYWFKSEhQaNGjbri8Xn33XcVEhKibt26ea1/+eWX+tOf/qRmzZopLCxMMTExGjx4sNd1MU899ZTi4+MlSY8++qgcDocaNWqkO+64Q++//76+/PJLORwOz/pFLpdL48ePV5MmTeR0OtWgQQM99thjcrlcV31cJOmuu+7Se++9J/6YPVBWsN0DALg2gwcPVmJiop555hnPL7ZJkyZp3LhxGjJkiB588EEVFBToxRdfVLdu3bRz505FR0era9eucrvd2rBhg+69915JUm5urqpVq6bc3FzP/nfu3Kni4mKvX/gvvPCC7rvvPiUnJ+v8+fNauHChBg8erGXLlqlv375e861Zs0aLFy/WmDFjFBsbq0aNGunkyZO688479cMPP2js2LEKDw/Xq6++qrCwMK+vzc/PV69evVS7dm2NHTtW0dHRysvL05IlS654XDZu3KikpCRVr17da33r1q3auHGjhg4dqri4OOXl5WnmzJm644479PHHH+uGG27QoEGDFB0drYcffljDhg1Tnz59VKNGDYWHh6uwsFDHjh3Tc889J0mqUaOGpB/Pitx3333asGGD/vjHP6pFixbau3evnnvuOR08eLDMhbjlHZeL2rdvr+eee0779+9XUlLSFb9XoEqxAASE8ePHW5KsYcOGea3n5eVZQUFB1qRJk7zW9+7dawUHB3vWS0tLrcjISOuxxx6zLMuy3G63FRMTYw0ePNgKCgqyzpw5Y1mWZWVnZ1vVqlWzTp8+7dnX999/77Xv8+fPW0lJSVb37t291iVZ1apVs/bv3++1np6ebkmyPvroI89afn6+FRUVZUmyDh8+bFmWZS1dutSSZG3duvUaj45lxcXFWb/97W/LrF86u2VZ1qZNmyxJ1htvvOFZO3z4sCXJmjp1qte2ffv2teLj48vsY968eVa1atWs3Nxcr/VZs2ZZkqz//ve/nrXLHZeLNm7caEmyFi1a9LPfI1AV8dIPEGAeeughr/tLliyR2+3WkCFD9M0333hudevWVWJiotauXStJqlatmm677TatX79ekvTJJ5/o1KlTGjt2rCzL0qZNmyT9eJYlKSnJ61qNn575OH36tAoLC9W1a1ft2LGjzHy33367WrZs6bX273//W7feeqs6duzoWatdu7aSk5O9trv4nMuWLdOFCxeu6bicOnVKNWvWLLP+09kvXLigU6dOqUmTJoqOji53/qv11ltvqUWLFmrevLnXce/evbskeY77ReUdl4suzv3NN9/4PA9QWREqQIBJSEjwun/o0CFZlqXExETVrl3b6/bJJ58oPz/fs23Xrl21fft2nT17Vrm5uapXr57atWuntm3bel7+2bBhg7p27er1HMuWLdOtt96q0NBQ1apVS7Vr19bMmTNVWFh4xfmkH68TSUxMLLPerFkzr/u33367fvvb32rChAmKjY1V//79lZOTU+aaj8uxyrnG4+zZs3ryySfVoEEDOZ1OxcbGqnbt2vruu+/Knf9qHTp0SPv37y9zzJs2bSpJXsddKv+4XDq3w+HweR6gsuIaFSDAXHpdh9vtlsPh0PLlyxUUFFRm+4vXVEhSly5ddOHCBW3atEm5ubmeIOnatatyc3P16aefqqCgwCtUcnNzdd9996lbt256+eWXVa9ePVWvXl05OTmaP3/+Fee7Fg6HQ2+//bY2b96s9957TytWrNCoUaM0bdo0bd682et7uVRMTIxOnz5dZv3Pf/6zcnJylJ6erk6dOnk+zG3o0KHlXsx7tdxut1q3bq3s7OxyH2/QoIHX/Z87Lhfnjo2N9XkeoLIiVIAA17hxY1mWpYSEBM//zV9Ox44dFRISotzcXOXm5urRRx+VJHXr1k2zZ8/W6tWrPfcveueddxQaGqoVK1bI6XR61nNycq56xvj4eB06dKjM+oEDB8rd/tZbb9Wtt96qSZMmaf78+UpOTtbChQv14IMPXvY5mjdvrsOHD5dZf/vtt5WSkqJp06Z51s6dO+f17qefc7mzHI0bN9bu3bvVo0eP6z4TcnHuFi1aXNd+gMqIl36AADdo0CAFBQVpwoQJZV76sCxLp06d8twPDQ3VzTffrAULFujIkSNeZ1TOnj2r6dOnq3HjxqpXr57na4KCguRwOFRaWupZy8vLu6aPl+/Tp482b96sLVu2eNYKCgr05ptvem13+vTpMt/DTTfdJElXfPmnU6dO2rdvX5ntgoKCyuzzxRdf9Pp+fs7Fd/5casiQITp+/Lhmz55d5rGzZ89e02fcbN++XVFRUWrVqtVVfw1QVXBGBQhwjRs31tNPP63MzEzl5eVpwIABioiI0OHDh7V06VL98Y9/1COPPOLZvmvXrpo8ebKioqLUunVrSVKdOnXUrFkzHThwQA888IDX/vv27avs7GzdfffdGj58uPLz8zVjxgw1adJEe/bsuaoZH3vsMc2bN09333230tLSPG9Pjo+P99rH66+/rpdfflkDBw5U48aNdebMGc2ePVuRkZHq06fPzz5H//79NXHiRK1bt069evXyrN97772aN2+eoqKi1LJlS23atEmrVq1STEzMVc3evn17LVq0SBkZGbr55ptVo0YN9evXT7///e+1ePFiPfTQQ1q7dq06d+6s0tJSffrpp1q8eLFWrFihDh06XNVzrFy5Uv369eMaFaA8tr3fCMA1ufj25IKCgnIff+edd6wuXbpY4eHhVnh4uNW8eXMrNTXVOnDggNd277//viXJuueee7zWH3zwQUuSNWfOnDL7njNnjpWYmGg5nU6refPmVk5Ojmeen5Jkpaamljvfnj17rNtvv90KDQ21fvWrX1kTJ0605syZ4/X25B07dljDhg2zGjZsaDmdTqtOnTrWvffea23btu2qjlGbNm2s0aNHe62dPn3aGjlypBUbG2vVqFHD6t27t/Xpp59a8fHxVkpKime7y709ubi42Bo+fLgVHR1tSfJ6q/L58+etKVOmWK1atbKcTqdVs2ZNq3379taECROswsLCqzoun3zyiSXJWrVq1VV9j0BV47AsPgoRQOUwb948paam6siRI7/IR+H7Q3p6utavX6/t27dzRgUoB6ECoNJwu91q06aNhg0bpieeeMLuca7o1KlTio+P1+LFi6/40hZQVREqAADAWLzrBwAAGItQAQAAxiJUAACAsQgVAABgrIAOFcuyVFRUVO4fIgMAAIEvoEPlzJkzioqK0pkzZ+weBQAAVICADhUAAFC5ESoAAMBYhAoAADAWoQIAAIxFqAAAAGMRKgAAwFiECgAAMBahAgAAjEWoAAAAYxEqAADAWIQKAAAwFqECAACMZWuoPPXUU3I4HF635s2b2zkSAAAwSLDdA7Rq1UqrVq3y3A8Otn0kAABgCNurIDg4WHXr1rV7DAAAYCDbr1E5dOiQ6tevr1//+tdKTk7WkSNHLruty+VSUVGR1w0AAFReDsuyLLuefPny5SouLlazZs104sQJTZgwQcePH9e+ffsUERFRZvunnnpKEyZMKLNeWFioyMhIv8/XaOz7ft+nJOVN7lsh+wUAoLKxNVQu9d133yk+Pl7Z2dkaPXp0mcddLpdcLpfnflFRkRo0aECoAABQSdl+jcpPRUdHq2nTpvrss8/KfdzpdMrpdP7CUwEAALvYfo3KTxUXF+vzzz9XvXr17B4FAAAYwNZQeeSRR7Ru3Trl5eVp48aNGjhwoIKCgjRs2DA7xwIAAIaw9aWfY8eOadiwYTp16pRq166tLl26aPPmzapdu7adYwEAAEPYGioLFy608+kBAIDhjLpGBQAA4KcIFQAAYCxCBQAAGItQAQAAxiJUAACAsQgVAABgLEIFAAAYi1ABAADGIlQAAICxCBUAAGAsQgUAABiLUAEAAMYiVAAAgLEIFQAAYCxCBQAAGItQAQAAxiJUAACAsQgVAABgLEIFAAAYi1ABAADGIlQAAICxCBUAAGAsQgUAABiLUAEAAMYiVAAAgLEIFQAAYCxCBQAAGItQAQAAxiJUAACAsQgVAABgLEIFAAAYi1ABAADGIlQAAICxCBUAAGAsQgUAABiLUAEAAMYiVAAAgLEIFQAAYCxCBQAAGItQAQAAxiJUAACAsQgVAABgLEIFAAAYi1ABAADGIlQAAICxCBUAAGAsQgUAABiLUAEAAMYiVAAAgLEIFQAAYCxCBQAAGItQAQAAxiJUAACAsQgVAABgLEIFAAAYi1ABAADGIlQAAICxCBUAAGAsQgUAABiLUAEAAMYyJlQmT54sh8Oh9PR0u0cBAACGMCJUtm7dqldeeUVt2rSxexQAAGAQ20OluLhYycnJmj17tmrWrGn3OAAAwCC2h0pqaqr69u2rnj17XnFbl8uloqIirxsAAKi8gu188oULF2rHjh3aunXrVW2flZWlCRMmVPBUAADAFLadUTl69KjS0tL05ptvKjQ09Kq+JjMzU4WFhZ7b0aNHK3hKAABgJ9vOqGzfvl35+flq166dZ620tFTr16/XSy+9JJfLpaCgIK+vcTqdcjqdv/SoAADAJraFSo8ePbR3716vtZEjR6p58+Z6/PHHy0QKAACoemwLlYiICCUlJXmthYeHKyYmpsw6AACommx/1w8AAMDl2Pqun0t9+OGHdo8AAAAMwhkVAABgLEIFAAAYi1ABAADGIlQAAICxCBUAAGAsQgUAABiLUAEAAMYiVAAAgLEIFQAAYCxCBQAAGItQAQAAxiJUAACAsQgVAABgLEIFAAAYi1ABAADGIlQAAICxCBUAAGAsQgUAABiLUAEAAMYiVAAAgLEIFQAAYCxCBQAAGItQAQAAxiJUAACAsQgVAABgLEIFAAAYi1ABAADGIlQAAICxCBUAAGAsQgUAABiLUAEAAMYiVAAAgLEIFQAAYCxCBQAAGItQAQAAxiJUAACAsQgVAABgLEIFAAAYi1ABAADGIlQAAICxCBUAAGAsQgUAABiLUAEAAMYiVAAAgLEIFQAAYCxCBQAAGItQAQAAxiJUAACAsQgVAABgLEIFAAAYi1ABAADGIlQAAICxCBUAAGAsQgUAABiLUAEAAMYiVAAAgLEIFQAAYCyfQuWLL77w9xwAAABl+BQqTZo00Z133ql//OMfOnfunL9nAgAAkORjqOzYsUNt2rRRRkaG6tatq//5n//Rli1brnk/M2fOVJs2bRQZGanIyEh16tRJy5cv92UkAABQCfkUKjfddJNeeOEFffXVV3rttdd04sQJdenSRUlJScrOzlZBQcFV7ScuLk6TJ0/W9u3btW3bNnXv3l39+/fX/v37fRkLAABUMg7Lsqzr3YnL5dLLL7+szMxMnT9/XiEhIRoyZIimTJmievXqXdO+atWqpalTp2r06NFX3LaoqEhRUVEqLCxUZGSkr+NfVqOx7/t9n5KUN7lvhewXAIDK5rre9bNt2zb96U9/Ur169ZSdna1HHnlEn3/+uVauXKmvvvpK/fv3v+p9lZaWauHChSopKVGnTp3K3cblcqmoqMjrBgAAKq9gX74oOztbOTk5OnDggPr06aM33nhDffr0UbVqP3ZPQkKC5s6dq0aNGl1xX3v37lWnTp107tw51ahRQ0uXLlXLli3L3TYrK0sTJkzwZWQAABCAfHrpJzExUaNGjdIDDzxw2Zd2zp8/rwULFiglJeVn93X+/HkdOXJEhYWFevvtt/X3v/9d69atKzdWXC6XXC6X535RUZEaNGjASz8AAFRSfrlGxZ969uypxo0b65VXXrnitlyjAgBA5ebTNSo5OTl66623yqy/9dZbev31169rILfb7XXWBAAAVF0+hUpWVpZiY2PLrNepU0fPPPPMVe8nMzNT69evV15envbu3avMzEx9+OGHSk5O9mUsAABQyfh0Me2RI0eUkJBQZj0+Pl5Hjhy56v3k5+fr/vvv14kTJxQVFaU2bdpoxYoVuuuuu3wZCwAAVDI+hUqdOnW0Z8+eMu/q2b17t2JiYq56P3PmzPHl6QEAQBXh00s/w4YN01/+8hetXbtWpaWlKi0t1Zo1a5SWlqahQ4f6e0YAAFBF+XRGZeLEicrLy1OPHj0UHPzjLtxut+6///5rukYFAADg5/gUKiEhIVq0aJEmTpyo3bt3KywsTK1bt1Z8fLy/5wMAAFWYT6FyUdOmTdW0aVN/zQIAAODFp1ApLS3V3LlztXr1auXn58vtdns9vmbNGr8MBwAAqjafQiUtLU1z585V3759lZSUJIfD4e+5AAAAfAuVhQsXavHixerTp4+/5wEAAPDw6e3JISEhatKkib9nAQAA8OJTqPz1r3/VCy+8IMP+niEAAKhkfHrpZ8OGDVq7dq2WL1+uVq1aqXr16l6PL1myxC/DAQCAqs2nUImOjtbAgQP9PQsAAIAXn0IlJyfH33MAAACU4dM1KpL0ww8/aNWqVXrllVd05swZSdJXX32l4uJivw0HAACqNp/OqHz55Ze6++67deTIEblcLt11112KiIjQlClT5HK5NGvWLH/PCQAAqiCfzqikpaWpQ4cOOn36tMLCwjzrAwcO1OrVq/02HAAAqNp8OqOSm5urjRs3KiQkxGu9UaNGOn78uF8GAwAA8OmMitvtVmlpaZn1Y8eOKSIi4rqHAgAAkHwMlV69eun555/33Hc4HCouLtb48eP5WH0AAOA3Pr30M23aNPXu3VstW7bUuXPnNHz4cB06dEixsbFasGCBv2cEAABVlE+hEhcXp927d2vhwoXas2ePiouLNXr0aCUnJ3tdXAsAAHA9fAoVSQoODtaIESP8OQsAAIAXn0LljTfe+NnH77//fp+GAQAA+CmfQiUtLc3r/oULF/T9998rJCREN9xwA6ECAAD8wqd3/Zw+fdrrVlxcrAMHDqhLly5cTAsAAPzG57/1c6nExERNnjy5zNkWAAAAX/ktVKQfL7D96quv/LlLAABQhfl0jcq//vUvr/uWZenEiRN66aWX1LlzZ78MBgAA4FOoDBgwwOu+w+FQ7dq11b17d02bNs0fcwEAAPgWKm63299zAAAAlOHXa1QAAAD8yaczKhkZGVe9bXZ2ti9PAQAA4Fuo7Ny5Uzt37tSFCxfUrFkzSdLBgwcVFBSkdu3aebZzOBz+mRIAAFRJPoVKv379FBERoddff101a9aU9OOHwI0cOVJdu3bVX//6V78OCQAAqiaHZVnWtX7Rr371K/3nP/9Rq1atvNb37dunXr16/WKfpVJUVKSoqCgVFhYqMjLS7/tvNPZ9v+9TkvIm962Q/QIAUNn4dDFtUVGRCgoKyqwXFBTozJkz1z0UAACA5GOoDBw4UCNHjtSSJUt07NgxHTt2TO+8845Gjx6tQYMG+XtGAABQRfl0jcqsWbP0yCOPaPjw4bpw4cKPOwoO1ujRozV16lS/DggAAKoun65RuaikpESff/65JKlx48YKDw/322BXg2tUAACo3K7rA99OnDihEydOKDExUeHh4bqO5gEAACjDp1A5deqUevTooaZNm6pPnz46ceKEJGn06NG8NRkAAPiNT6Hy8MMPq3r16jpy5IhuuOEGz/rvfvc7ffDBB34bDgAAVG0+XUz7n//8RytWrFBcXJzXemJior788ku/DAYAAODTGZWSkhKvMykXffvtt3I6ndc9FAAAgORjqHTt2lVvvPGG577D4ZDb7dazzz6rO++802/DAQCAqs2nl36effZZ9ejRQ9u2bdP58+f12GOPaf/+/fr222/13//+198zAgCAKsqnMypJSUk6ePCgunTpov79+6ukpESDBg3Szp071bhxY3/PCAAAqqhrPqNy4cIF3X333Zo1a5aeeOKJipgJAABAkg9nVKpXr649e/ZUxCwAAABefHrpZ8SIEZozZ46/ZwEAAPDi08W0P/zwg1577TWtWrVK7du3L/M3frKzs/0yHAAAqNquKVS++OILNWrUSPv27VO7du0kSQcPHvTaxuFw+G86AABQpV1TqCQmJurEiRNau3atpB8/Mn/69Om68cYbK2Q4AABQtV3TNSqX/nXk5cuXq6SkxK8DAQAAXOTTxbQXXRouAAAA/nRNoeJwOMpcg8I1KQAAoKJc0zUqlmXpgQce8PzhwXPnzumhhx4q866fJUuW+G9CAABQZV1TqKSkpHjdHzFihF+HAQAA+KlrCpWcnJyKmgMAAKCM67qYFgAAoCIRKgAAwFi2hkpWVpZuvvlmRUREqE6dOhowYIAOHDhg50gAAMAgtobKunXrlJqaqs2bN2vlypW6cOGCevXqxYfIAQAAST7+UUJ/+eCDD7zuz507V3Xq1NH27dvVrVu3Mtu7XC65XC7P/aKiogqfEQAA2MfWULlUYWGhJKlWrVrlPp6VlaUJEyb8kiNViEZj36+wfedN7lth+wZ+afy3gsvhZ6PqMOZiWrfbrfT0dHXu3FlJSUnlbpOZmanCwkLP7ejRo7/wlAAA4JdkzBmV1NRU7du3Txs2bLjsNk6n0/OpuAAAoPIzIlTGjBmjZcuWaf369YqLi7N7HAAAYAhbQ8WyLP35z3/W0qVL9eGHHyohIcHOcQAAgGFsDZXU1FTNnz9f//znPxUREaGTJ09KkqKiohQWFmbnaAAAwAC2Xkw7c+ZMFRYW6o477lC9evU8t0WLFtk5FgAAMITtL/0AAABcjjFvTwYAALgUoQIAAIxFqAAAAGMRKgAAwFiECgAAMBahAgAAjEWoAAAAYxEqAADAWIQKAAAwFqECAACMRagAAABjESoAAMBYhAoAADAWoQIAAIxFqAAAAGMRKgAAwFiECgAAMBahAgAAjEWoAAAAYxEqAADAWIQKAAAwFqECAACMRagAAABjESoAAMBYhAoAADAWoQIAAIxFqAAAAGMRKgAAwFiECgAAMBahAgAAjEWoAAAAYxEqAADAWIQKAAAwFqECAACMRagAAABjESoAAMBYhAoAADAWoQIAAIxFqAAAAGMRKgAAwFiECgAAMBahAgAAjEWoAAAAYxEqAADAWIQKAAAwFqECAACMRagAAABjESoAAMBYhAoAADAWoQIAAIxFqAAAAGMRKgAAwFiECgAAMBahAgAAjEWoAAAAYxEqAADAWIQKAAAwFqECAACMRagAAABj2Roq69evV79+/VS/fn05HA69++67do4DAAAMY2uolJSUqG3btpoxY4adYwAAAEMF2/nk99xzj+655x47RwAAAAazNVSulcvlksvl8twvKiqycRoAAFDRAipUsrKyNGHCBLvHMFqjse9XyH7zJvetkP1KFTdzRQrE41GRMwOwV0X+O2r3vx0B9a6fzMxMFRYWem5Hjx61eyQAAFCBAuqMitPplNPptHsMAADwCwmoMyoAAKBqsfWMSnFxsT777DPP/cOHD2vXrl2qVauWGjZsaONkAADABLaGyrZt23TnnXd67mdkZEiSUlJSNHfuXJumAgAAprA1VO644w5ZlmXnCAAAwGBcowIAAIxFqAAAAGMRKgAAwFiECgAAMBahAgAAjEWoAAAAYxEqAADAWIQKAAAwFqECAACMRagAAABjESoAAMBYhAoAADAWoQIAAIxFqAAAAGMRKgAAwFiECgAAMBahAgAAjEWoAAAAYxEqAADAWIQKAAAwFqECAACMRagAAABjESoAAMBYhAoAADAWoQIAAIxFqAAAAGMRKgAAwFiECgAAMBahAgAAjEWoAAAAYxEqAADAWIQKAAAwFqECAACMRagAAABjESoAAMBYhAoAADAWoQIAAIxFqAAAAGMRKgAAwFiECgAAMBahAgAAjEWoAAAAYxEqAADAWIQKAAAwFqECAACMRagAAABjESoAAMBYhAoAADAWoQIAAIxFqAAAAGMRKgAAwFiECgAAMBahAgAAjEWoAAAAYxEqAADAWIQKAAAwFqECAACMRagAAABjESoAAMBYRoTKjBkz1KhRI4WGhuqWW27Rli1b7B4JAAAYwPZQWbRokTIyMjR+/Hjt2LFDbdu2Ve/evZWfn2/3aAAAwGa2h0p2drb+8Ic/aOTIkWrZsqVmzZqlG264Qa+99prdowEAAJsF2/nk58+f1/bt25WZmelZq1atmnr27KlNmzaV2d7lcsnlcnnuFxYWSpKKiooqZD636/sK2W8gqqhjLAXmcQ7E41GRM1eUivzZCMTjgf/Dz4a3QD0eERERcjgcP7+RZaPjx49bkqyNGzd6rT/66KNWx44dy2w/fvx4SxI3bty4cePGrRLcCgsLr9gKtp5RuVaZmZnKyMjw3He73fr2228VExNz5SK7RkVFRWrQoIGOHj2qyMhIv+67quAY+gfH0T84jtePY+gfHMf/ExERccVtbA2V2NhYBQUF6euvv/Za//rrr1W3bt0y2zudTjmdTq+16OjoihxRkZGRVf4H6XpxDP2D4+gfHMfrxzH0D47j1bH1YtqQkBC1b99eq1ev9qy53W6tXr1anTp1snEyAABgAttf+snIyFBKSoo6dOigjh076vnnn1dJSYlGjhxp92gAAMBmtofK7373OxUUFOjJJ5/UyZMnddNNN+mDDz7QjTfeaOtcTqdT48ePL/NSE64ex9A/OI7+wXG8fhxD/+A4XhuHZVmW3UMAAACUx/YPfAMAALgcQgUAABiLUAEAAMYiVAAAgLEIFQAAYCxCpRwzZsxQo0aNFBoaqltuuUVbtmyxe6SAkpWVpZtvvlkRERGqU6eOBgwYoAMHDtg9VkCbPHmyHA6H0tPT7R4l4Bw/flwjRoxQTEyMwsLC1Lp1a23bts3usQJKaWmpxo0bp4SEBIWFhalx48aaOHGieNPoz1u/fr369eun+vXry+Fw6N133/V63LIsPfnkk6pXr57CwsLUs2dPHTp0yJ5hDUaoXGLRokXKyMjQ+PHjtWPHDrVt21a9e/dWfn6+3aMFjHXr1ik1NVWbN2/WypUrdeHCBfXq1UslJSV2jxaQtm7dqldeeUVt2rSxe5SAc/r0aXXu3FnVq1fX8uXL9fHHH2vatGmqWbOm3aMFlClTpmjmzJl66aWX9Mknn2jKlCl69tln9eKLL9o9mtFKSkrUtm1bzZgxo9zHn332WU2fPl2zZs3SRx99pPDwcPXu3Vvnzp37hSc1nF/+DHIl0rFjRys1NdVzv7S01Kpfv76VlZVl41SBLT8/35JkrVu3zu5RAs6ZM2esxMREa+XKldbtt99upaWl2T1SQHn88cetLl262D1GwOvbt681atQor7VBgwZZycnJNk0UeCRZS5cu9dx3u91W3bp1ralTp3rWvvvuO8vpdFoLFiywYUJzcUblJ86fP6/t27erZ8+enrVq1aqpZ8+e2rRpk42TBbbCwkJJUq1atWyeJPCkpqaqb9++Xj+TuHr/+te/1KFDBw0ePFh16tTRb37zG82ePdvusQLObbfdptWrV+vgwYOSpN27d2vDhg265557bJ4scB0+fFgnT570+m87KipKt9xyC79vLmH7R+ib5JtvvlFpaWmZj++/8cYb9emnn9o0VWBzu91KT09X586dlZSUZPc4AWXhwoXasWOHtm7davcoAeuLL77QzJkzlZGRof/93//V1q1b9Ze//EUhISFKSUmxe7yAMXbsWBUVFal58+YKCgpSaWmpJk2apOTkZLtHC1gnT56UpHJ/31x8DD8iVFChUlNTtW/fPm3YsMHuUQLK0aNHlZaWppUrVyo0NNTucQKW2+1Whw4d9Mwzz0iSfvOb32jfvn2aNWsWoXINFi9erDfffFPz589Xq1attGvXLqWnp6t+/focR1Q4Xvr5idjYWAUFBenrr7/2Wv/6669Vt25dm6YKXGPGjNGyZcu0du1axcXF2T1OQNm+fbvy8/PVrl07BQcHKzg4WOvWrdP06dMVHBys0tJSu0cMCPXq1VPLli291lq0aKEjR47YNFFgevTRRzV27FgNHTpUrVu31u9//3s9/PDDysrKsnu0gHXxdwq/b66MUPmJkJAQtW/fXqtXr/asud1urV69Wp06dbJxssBiWZbGjBmjpUuXas2aNUpISLB7pIDTo0cP7d27V7t27fLcOnTooOTkZO3atUtBQUF2jxgQOnfuXOat8QcPHlR8fLxNEwWm77//XtWqef+6CAoKktvttmmiwJeQkKC6det6/b4pKirSRx99xO+bS/DSzyUyMjKUkpKiDh06qGPHjnr++edVUlKikSNH2j1awEhNTdX8+fP1z3/+UxEREZ7XW6OiohQWFmbzdIEhIiKizDU94eHhiomJ4Vqfa/Dwww/rtttu0zPPPKMhQ4Zoy5YtevXVV/Xqq6/aPVpA6devnyZNmqSGDRuqVatW2rlzp7KzszVq1Ci7RzNacXGxPvvsM8/9w4cPa9euXapVq5YaNmyo9PR0Pf3000pMTFRCQoLGjRun+vXra8CAAfYNbSK733ZkohdffNFq2LChFRISYnXs2NHavHmz3SMFFEnl3nJycuweLaDx9mTfvPfee1ZSUpLldDqt5s2bW6+++qrdIwWcoqIiKy0tzWrYsKEVGhpq/frXv7aeeOIJy+Vy2T2a0dauXVvuv4UpKSmWZf34FuVx48ZZN954o+V0Oq0ePXpYBw4csHdoAzksi48WBAAAZuIaFQAAYCxCBQAAGItQAQAAxiJUAACAsQgVAABgLEIFAAAYi1ABAADGIlQAAICxCBUAAGAsQgUAABiLUAEAAMb6f2UflbVpJN4uAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# @title rewards (after)\n",
    "\n",
    "from matplotlib import pyplot as plt\n",
    "df_results['rewards (after)'].plot(kind='hist', bins=20, title='rewards (after)')\n",
    "plt.gca().spines[['top', 'right',]].set_visible(False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e3c96b07-71f0-4636-ab36-d505f7fc7b7a",
   "metadata": {
    "id": "e3c96b07-71f0-4636-ab36-d505f7fc7b7a"
   },
   "source": [
    "### Save Aligned Model Objects"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "IKbx-t8husvX",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "id": "IKbx-t8husvX",
    "outputId": "b69c7f1a-f327-47c3-bf9a-bf12f8552c68"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.google.colaboratory.intrinsic+json": {
       "type": "string"
      },
      "text/plain": [
       "'aligned-gpt2-movie_reviewer'"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ALIGNED_MODEL_NAME"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "645c5b46-b4e7-4071-84ed-4e14f371e7f6",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "645c5b46-b4e7-4071-84ed-4e14f371e7f6",
    "outputId": "8507e0c9-5624-44b5-93b1-caa60488cdef"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('aligned-gpt2-movie_reviewer/tokenizer_config.json',\n",
       " 'aligned-gpt2-movie_reviewer/special_tokens_map.json',\n",
       " 'aligned-gpt2-movie_reviewer/vocab.json',\n",
       " 'aligned-gpt2-movie_reviewer/merges.txt',\n",
       " 'aligned-gpt2-movie_reviewer/added_tokens.json',\n",
       " 'aligned-gpt2-movie_reviewer/tokenizer.json')"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.save_pretrained(ALIGNED_MODEL_NAME)\n",
    "tokenizer.save_pretrained(ALIGNED_MODEL_NAME)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "bc1ed0ff-125e-4441-8b79-9405bd2a9d8a",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 116,
     "referenced_widgets": [
      "a1334367870e4c27b47d8638bacd1c8f",
      "da0598234e484de9872383106d5b45e2",
      "2645f71273704bc1823828adf4e31f9d",
      "16656c4fc26f48c3bbbcbbb94bbfd5c3",
      "a9d5de9af25048dda9553df10cf52ac9",
      "7ac268d459dd4ceb98237057ef63b5cf",
      "ab8ecce0350447e9898cd288be6654dd",
      "ee10b481e2ee44f4abf32639dbbd1919",
      "ea257116758f400da8751eef1886bbea",
      "50b466e09a1544c595221e94d08d68cf",
      "6506e794db9b41a8822f1e6744031979",
      "0e88f9acc6c84340a3524a95ed5cf1da",
      "df2678cc4bf74b01b5e0d7a223005d9e",
      "15d6c793a70644beb54c5e71c76aa900",
      "cf0f4ffec6604e9ca6fc9667ef4b32e2",
      "eca055c6a073422dbc6038850d90166d",
      "3001b740eca44b0b9741b3a4e4c4de0d",
      "60b5f4007d9640279a5de2949625fb2b",
      "5397dbf6312c400c816a8de45c7144a6",
      "d9bd270f150c4e8fab043cc2fe0478f3",
      "1cb383c5a22c46d5908be7c285f394f3",
      "ddaf8d2bf0fe40e8a1278017b76d0d11"
     ]
    },
    "id": "bc1ed0ff-125e-4441-8b79-9405bd2a9d8a",
    "outputId": "62180970-ed7b-4b25-d6a5-2dcad04dbbe1"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a1334367870e4c27b47d8638bacd1c8f",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "model.safetensors:   0%|          | 0.00/498M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0e88f9acc6c84340a3524a95ed5cf1da",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "README.md:   0%|          | 0.00/5.17k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.google.colaboratory.intrinsic+json": {
       "type": "string"
      },
      "text/plain": [
       "CommitInfo(commit_url='https://huggingface.co/raghavbali/aligned-gpt2-movie_reviewer/commit/2fcc93f09aad019ef27ccf064d447cbd50e6472b', commit_message='PPO alignment done', commit_description='', oid='2fcc93f09aad019ef27ccf064d447cbd50e6472b', pr_url=None, pr_revision=None, pr_num=None)"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "if SAVE_TRAINED_OBJECTS and HF_NAMESPACE_TO_PUSH_TO is not \"XXXXXX\":\n",
    "    hf_repo_id = f\"{HF_NAMESPACE_TO_PUSH_TO}/{ALIGNED_MODEL_NAME}\"\n",
    "    model.push_to_hub(\n",
    "        hf_repo_id,\n",
    "        commit_message=\"Add model object, PPO alignment done\")\n",
    "    tokenizer.push_to_hub(\n",
    "        hf_repo_id,\n",
    "        commit_message=\"Add tokenizer object, PPO alignment done\")\n",
    "else:\n",
    "    print(\"You have chosen not to push objects to any HF namespace/repo\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ff932ed8-2588-4a97-aed2-b027185a629e",
   "metadata": {
    "id": "ff932ed8-2588-4a97-aed2-b027185a629e"
   },
   "source": [
    "## Generate and Compare Aligned vs Non-Aligned Models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "e389057f-53aa-4071-8dd2-d3312cbaaa49",
   "metadata": {
    "id": "e389057f-53aa-4071-8dd2-d3312cbaaa49"
   },
   "outputs": [],
   "source": [
    "from IPython.display import display, Markdown"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "dab01114-b221-4cde-a84d-55bf4f250460",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "dab01114-b221-4cde-a84d-55bf4f250460",
    "outputId": "89d5c9d2-84ba-4a44-87d2-71e9c9a77f0d"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Some weights of the model checkpoint at ./aligned-gpt2-movie_reviewer were not used when initializing GPT2LMHeadModel: ['v_head.summary.bias', 'v_head.summary.weight']\n",
      "- This IS expected if you are initializing GPT2LMHeadModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",
      "- This IS NOT expected if you are initializing GPT2LMHeadModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n"
     ]
    }
   ],
   "source": [
    "hub_model = AutoModelForCausalLMWithValueHead.from_pretrained(f'./{ALIGNED_MODEL_NAME}').to(device) #,cache_dir=\"/workspace/\"\n",
    "# create a reference model\n",
    "hub_tokenizer = AutoTokenizer.from_pretrained(f'./{ALIGNED_MODEL_NAME}',) #cache_dir=\"/workspace/\"\n",
    "\n",
    "hub_tokenizer.pad_token = tokenizer.eos_token"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "e8fd2f48-1147-4bb4-95f0-6fc2342a2257",
   "metadata": {
    "id": "e8fd2f48-1147-4bb4-95f0-6fc2342a2257"
   },
   "outputs": [],
   "source": [
    "reviews = [\n",
    "    \"No big names\",\n",
    "    \"The director\",\n",
    "    \"What\",\n",
    "    \"Lame\",\n",
    "    \"Space invaders\",\n",
    "    \"Here are my 2 cents on the movie\",\n",
    "\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "2bb1c4e6-3a0c-4f0c-9649-e701d00591b2",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "2bb1c4e6-3a0c-4f0c-9649-e701d00591b2",
    "outputId": "ab4d3cd4-8dde-4037-ecd7-ae3760b3dc08"
   },
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "### Prompt: No big names..."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### ALIGNED-MODEL "
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "No big names in this movie, but I can't wait for it to come to see me again! There are plenty of things that are"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### NON- ALIGNED-MODEL "
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "No big names, but I really like the ending. I thought that was a bit too long and boring. This is a great story about"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "---"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "### Prompt: The director..."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### ALIGNED-MODEL "
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "The director, Martin Scorsese, plays the character of a man who goes missing from his home in Ireland. He is haunted by"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### NON- ALIGNED-MODEL "
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "The director of \"Black Mirror\" David Lynch knows what to do with his own dead brother. His film has become the subject of a"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "---"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "### Prompt: What..."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### ALIGNED-MODEL "
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "What I've seen on television is an absolutely stunning, moving, moving, and extremely entertaining story of the life of a beautiful young"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### NON- ALIGNED-MODEL "
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "What to do:\n",
       "\n",
       " of the new movies is really very good. I would recommend this film to anyone that likes the gritty"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "---"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "### Prompt: Lame..."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### ALIGNED-MODEL "
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "Lame with the money and the lack of a proper score. I liked the script and the characters, but I couldn't stomach the"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### NON- ALIGNED-MODEL "
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "Lame in the name of a good guy, but he's not a bad guy. He doesn't seem like a guy who has"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "---"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "### Prompt: Space invaders..."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### ALIGNED-MODEL "
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "Space invaders from the 20th Century, they were great films. I would recommend this to someone who is into horror and suspense. They"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### NON- ALIGNED-MODEL "
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "Space invaders, are a part of the film. The plot is based on a comic book which has been published as a comic book."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "---"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "### Prompt: Here are my 2 cents on the movie..."
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### ALIGNED-MODEL "
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "Here are my 2 cents on the movie. There are some great action and the characters are well crafted. Everything about this movie will be enjoyable and funny. The script"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "#### NON- ALIGNED-MODEL "
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "Here are my 2 cents on the movie. I was very disappointed in the performance of the story and the plot. The main character was really bad and I was impressed"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "---"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "for review in reviews:\n",
    "    inputs = hub_tokenizer(review, return_tensors=\"pt\", return_token_type_ids=False).to(device)\n",
    "    display(Markdown((f\"### Prompt: {review}...\")))\n",
    "    display(Markdown((\"#### ALIGNED-MODEL \")))\n",
    "    outputs = hub_model.generate(**inputs,max_new_tokens=25,temperature=0.8,do_sample=True, pad_token_id=tokenizer.eos_token_id)\n",
    "    display(Markdown((tokenizer.decode(outputs[0], skip_special_tokens=True))))\n",
    "    display(Markdown((\"#### NON- ALIGNED-MODEL \")))\n",
    "    outputs = ref_model.generate(**inputs, max_new_tokens=25,temperature=0.8,do_sample=True, pad_token_id=tokenizer.eos_token_id)\n",
    "    display(Markdown((tokenizer.decode(outputs[0], skip_special_tokens=True))))\n",
    "    display(Markdown((\"---\")))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cd3908f4-35b6-492d-b946-0fb3cc621596",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "gpuType": "T4",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "007d96ccd3ec436ab7b4e53589df8171": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_5c191aa20c87406f9b7e56c286d014f6",
      "placeholder": "​",
      "style": "IPY_MODEL_1e5981a6ac3e48a78a8528c6de2fc7f6",
      "value": "vocab.txt: 100%"
     }
    },
    "017d20c252904df1bdb48bda9c8c1465": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_030ab77c206e47c2884d8e3cfc811a36",
      "placeholder": "​",
      "style": "IPY_MODEL_474c6a730a1f4222b99e3b4eede9c8cd",
      "value": "Map: 100%"
     }
    },
    "02538b35512e40cc90c7da5c44d439f0": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "030ab77c206e47c2884d8e3cfc811a36": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "040ec7d31dc24b269cc6593d98d26fe8": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "059b7ac7e5e6451097d783e404cf7a7e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "05b8baecad7740f5834d453a84e82f4a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "089c84d7a2384a99b6ee278722bdb867": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "PasswordModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "PasswordModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "PasswordView",
      "continuous_update": true,
      "description": "Token:",
      "description_tooltip": null,
      "disabled": false,
      "layout": "IPY_MODEL_05b8baecad7740f5834d453a84e82f4a",
      "placeholder": "​",
      "style": "IPY_MODEL_97412f765b2942519106c00ffa2020ba",
      "value": ""
     }
    },
    "08ffbe2a042544aeb8565179a327c986": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "09591085118748e888a175fedd09e749": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_f2a33680c4ce4e8face4bbd17b2fec03",
       "IPY_MODEL_d4f74fade7c94c2c88911394fe3f78dd",
       "IPY_MODEL_8529ec8a1e594b84aca64f9ba6b5e5e0"
      ],
      "layout": "IPY_MODEL_5907f9914df3490da8af4435fdd47471"
     }
    },
    "096e494497254950b4c4eb2af68df123": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_34025be36a4647bcba0a87e6f901e9b4",
      "max": 231508,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_d42211410ee64e7b9fff9c3653dea5b0",
      "value": 231508
     }
    },
    "09e2170d4b814dec842545ffef6aa186": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "0a70d9f2928841a5ab5a71a1f90beca6": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0ba241098153433eb162fa4dcac0e6bc": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "0c73a3203dff4a949e7c15be100277c5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "0c75945a099a4f6a850a2cdedfbc1d23": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0de62df79d03478184c5c5413795e479": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0e88f9acc6c84340a3524a95ed5cf1da": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_df2678cc4bf74b01b5e0d7a223005d9e",
       "IPY_MODEL_15d6c793a70644beb54c5e71c76aa900",
       "IPY_MODEL_cf0f4ffec6604e9ca6fc9667ef4b32e2"
      ],
      "layout": "IPY_MODEL_eca055c6a073422dbc6038850d90166d"
     }
    },
    "0edf9f054c3c4e18976736d908a803f5": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0f5b75071a944a7490e2aa111c4f79f9": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_c84eb5f041614dc4bf577a45edc3b26d",
      "max": 333,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_39b263537a664391af6612bd18b1043b",
      "value": 333
     }
    },
    "10a80c2039bd411581fe1dd895ebaac9": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "11ce622e2580467d807fa8e7dd87e3b5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_eac93a970ece49f1b43969cde67d6619",
      "max": 735,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_9d5becf11dbe4b22a706147b799f2d51",
      "value": 735
     }
    },
    "1204c9dc382e45e4a4fd62c5be9f4274": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "124fa02529c7427593b3bf71d4612646": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_54a77ee7423f4d649a371bc4a2909b60",
      "placeholder": "​",
      "style": "IPY_MODEL_209090fc289c4eaeb2d69724e4141467",
      "value": " 25000/25000 [00:00&lt;00:00, 74474.05 examples/s]"
     }
    },
    "15d6c793a70644beb54c5e71c76aa900": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_5397dbf6312c400c816a8de45c7144a6",
      "max": 5174,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_d9bd270f150c4e8fab043cc2fe0478f3",
      "value": 5174
     }
    },
    "1648bfefa8d146cdad215588de744fac": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "16656c4fc26f48c3bbbcbbb94bbfd5c3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_50b466e09a1544c595221e94d08d68cf",
      "placeholder": "​",
      "style": "IPY_MODEL_6506e794db9b41a8822f1e6744031979",
      "value": " 498M/498M [00:15&lt;00:00, 39.5MB/s]"
     }
    },
    "16d37ae87d6046dcbb52e7acf3822e81": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_62643405eecf4af2bad191b99bffd84c",
       "IPY_MODEL_8a12f57eccaa4878a5f7d4f64234f02b",
       "IPY_MODEL_680643b191104f38adcb2aef4cf7d5c1"
      ],
      "layout": "IPY_MODEL_1204c9dc382e45e4a4fd62c5be9f4274"
     }
    },
    "17c551db5ae944a1ad9fe7fc38f88a9f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "190f641967bb4bad9706a642ac7c2da2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_91f8f77b67f34b6b8744b6018c3f2ed3",
      "placeholder": "​",
      "style": "IPY_MODEL_e55a650c0f6f48fdb643ac0c3ecb203f",
      "value": " 232k/232k [00:00&lt;00:00, 1.42MB/s]"
     }
    },
    "193ceb339ec1490587881933788b3015": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_cae1ddea6b0b4accb7a5ea9a320070ea",
      "max": 665,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_2bb6639cb150487ab78a7b0d2d54a3ff",
      "value": 665
     }
    },
    "1b9ef18c0fbf4ff39d74e2d54c107991": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "1c5621316bce482abcd926e7a053a9b0": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_80f799821f6145238fccc14826e56e8e",
      "placeholder": "​",
      "style": "IPY_MODEL_f5a4aa9b9275450e8d3f028b5ee4d4ce",
      "value": " 1.36M/1.36M [00:00&lt;00:00, 4.11MB/s]"
     }
    },
    "1cb383c5a22c46d5908be7c285f394f3": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1dd60f1ebabe4d74994a51b688c73b80": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1e5981a6ac3e48a78a8528c6de2fc7f6": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "1f038ef1ec8b4d1e88654b68178406f4": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1f1f3151f6cf4b3da646f13253f5fbb1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_477ee2c2aaa34dddab41b56142cd9659",
      "max": 7809,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_2c01a48af848479093a70016a13d24e8",
      "value": 7809
     }
    },
    "209090fc289c4eaeb2d69724e4141467": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "20ba4fcb1fe0441d9f8250904390a1bf": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f459c938c12542c3837b019c0770a760",
      "placeholder": "​",
      "style": "IPY_MODEL_dcb3e035278646df81517bcbd63ad671",
      "value": ""
     }
    },
    "21168c6da7434941a8062e727a8acb74": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "24d1ae4c531f44549a69cbf13435b155": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "26438f4a85cb4e05923cbd1444df2364": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_dc0b9472eea14c7bbd285bef3d044583",
      "placeholder": "​",
      "style": "IPY_MODEL_373c0550a0e641f1a02ae3b4e14440e8",
      "value": "tokenizer.json: 100%"
     }
    },
    "2645f71273704bc1823828adf4e31f9d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ee10b481e2ee44f4abf32639dbbd1919",
      "max": 497780540,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_ea257116758f400da8751eef1886bbea",
      "value": 497780540
     }
    },
    "274fa2d33c0247c1bb584751a64e28a7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a09a7a2ff8804931b7ad83ee773e0417",
      "max": 2410,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_f7037dc6dd074cc483e6482bea4581c1",
      "value": 2410
     }
    },
    "29ce0e9095b646c3b50d721559af6bf6": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ButtonStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ButtonStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "button_color": null,
      "font_weight": ""
     }
    },
    "2ac034bd2c6d44d6aad88f8af57d3f13": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "2b8e1447938d4c2592da8096f70a0310": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "2bb6639cb150487ab78a7b0d2d54a3ff": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "2bc64794e64d4cdfbd5222651a469857": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_9ebfe425886c46958067b0ce4aac1980",
      "placeholder": "​",
      "style": "IPY_MODEL_2ce562645c3c43549851426f8c98bcd8",
      "value": "unsupervised-00000-of-00001.parquet: 100%"
     }
    },
    "2bf0912777294d7384ac246fca984f94": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_3834ba5c3e1b49bc9e03a18c37a2d931",
      "placeholder": "​",
      "style": "IPY_MODEL_3f6f527d09d54f7f9e38a465dbe04af6",
      "value": "test-00000-of-00001.parquet: 100%"
     }
    },
    "2c01a48af848479093a70016a13d24e8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "2caaf115ae0543a7b3cec697e73dfa48": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_601f9cc9a5794be48c4df19677e3ba27",
      "placeholder": "​",
      "style": "IPY_MODEL_10a80c2039bd411581fe1dd895ebaac9",
      "value": " 466k/466k [00:00&lt;00:00, 1.94MB/s]"
     }
    },
    "2cab9533d3c1442a90872341972df272": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e454660115b240f1aea38f25d04012e0",
      "max": 1355256,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_67e45ac4538d4183b1eea084e899dd48",
      "value": 1355256
     }
    },
    "2cce25a349ce479fbb0c8ee33c88eb6c": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2ce562645c3c43549851426f8c98bcd8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "2e753528edde49d8a50c57ef204ac5b3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_875ffc1310844528b287d4a257cf5f97",
      "placeholder": "​",
      "style": "IPY_MODEL_7fee544e10ba4b399bae8c1a68c16e5b",
      "value": "\n<b>Pro Tip:</b> If you don't already have one, you can create a dedicated\n'notebooks' token with 'write' access, that you can then easily reuse for all\nnotebooks. </center>"
     }
    },
    "2fb4cb7c8c6f44a2a5cc4f8badde0b9a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "3001b740eca44b0b9741b3a4e4c4de0d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "300353225bed4b309dcc5e1df1d1afa0": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "30844b55fec7479d82802a69b036eb1f": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "318fd3adc242499885c56930f147566d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_8ca44fcac9be4aaab123128bd6743e68",
       "IPY_MODEL_bd2c9ebf67a04a538f3bde44ce8107f3",
       "IPY_MODEL_b75b711dc88f43f9b757697de1bdcf90"
      ],
      "layout": "IPY_MODEL_8de59c241b574fcb83b8036ca44fef17"
     }
    },
    "33a3ca1767f94716bfa274e977c43215": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "34025be36a4647bcba0a87e6f901e9b4": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "369fac0bbb324623acc8f2a9ad3c211e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "373c0550a0e641f1a02ae3b4e14440e8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "3758d7613cca4c6b9309e7e991cf03f5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_816feb637b724bca83ce7347d0b94547",
      "placeholder": "​",
      "style": "IPY_MODEL_5f54bf184d3d4e8687543ecd27afc3cc",
      "value": " 20.5M/20.5M [00:00&lt;00:00, 169MB/s]"
     }
    },
    "3834ba5c3e1b49bc9e03a18c37a2d931": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "39b263537a664391af6612bd18b1043b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "3b1c47003ad54aeaa3bfeae65a71ffc6": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_7b6bbb45ed2d4f77a848b6ba29d2f23b",
      "placeholder": "​",
      "style": "IPY_MODEL_ca8c415219f4467b92c2d06ffa182c67",
      "value": " 2410/2410 [00:01&lt;00:00, 1829.18 examples/s]"
     }
    },
    "3bd73db8e99a42c2ad84ec04f32e8113": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "3c426c71a2d147c1af9330ce0703e14d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "3e7a016083d44af6a1d8b047b803cdb0": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "3f17fc1b5c2c4f1996151cbded670ae3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "3f6f527d09d54f7f9e38a465dbe04af6": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "408b38405e21448fb939f2609e8c2cea": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_66eaee16e9ec490086dffd09369ccf60",
      "placeholder": "​",
      "style": "IPY_MODEL_f260f5c57a2948ca82c3862639405f13",
      "value": " 7.81k/7.81k [00:00&lt;00:00, 558kB/s]"
     }
    },
    "408f2784f32d429099762d55c13b6fdd": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_4b2a26e21c8548f29fe0cde2b1159dc4",
       "IPY_MODEL_8c3269edbb0148259348b37cfd2ee694",
       "IPY_MODEL_2caaf115ae0543a7b3cec697e73dfa48"
      ],
      "layout": "IPY_MODEL_9561bbe794774920a2594c87cc2822b7"
     }
    },
    "410adfcb106444c4a5d52f9490788e7d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_905482147fcc4b0eadad09ec99333097",
       "IPY_MODEL_193ceb339ec1490587881933788b3015",
       "IPY_MODEL_75ada9f5c4d9466e8a42597ea9ac6466"
      ],
      "layout": "IPY_MODEL_806a776295c44eb7998dca7ace00c6cf"
     }
    },
    "42cdd539bdcd4563ac7263ca3a87a795": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_33a3ca1767f94716bfa274e977c43215",
      "max": 456318,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_5d75918be7df4027a56f972503bd8b78",
      "value": 456318
     }
    },
    "42f8b2a5cc8845c489dadaade07e35df": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4341e013b9dd4c87ac30d7e74978834f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_1648bfefa8d146cdad215588de744fac",
      "placeholder": "​",
      "style": "IPY_MODEL_2b8e1447938d4c2592da8096f70a0310",
      "value": " 25000/25000 [00:00&lt;00:00, 94075.88 examples/s]"
     }
    },
    "45ca8ac26b964d05bb5e42cdd31a012a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "46351724f1154e6892c089960a4b40a9": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "474c6a730a1f4222b99e3b4eede9c8cd": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "477ee2c2aaa34dddab41b56142cd9659": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "47c77d548220417abec9f72df138401a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "490eddc9ab7048e8bfdc15ba8f5fa6a7": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "493432db1fb84848964d14966627fc4b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_46351724f1154e6892c089960a4b40a9",
      "placeholder": "​",
      "style": "IPY_MODEL_300353225bed4b309dcc5e1df1d1afa0",
      "value": " 42.0M/42.0M [00:00&lt;00:00, 186MB/s]"
     }
    },
    "49b4cd3787ef47399b90c14a627a67da": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e76ba5f1a6ce49229bee1f51679d1efc",
      "placeholder": "​",
      "style": "IPY_MODEL_fa2b4efbbfe0442580a85b0efae75d90",
      "value": " 735/735 [00:00&lt;00:00, 40.8kB/s]"
     }
    },
    "4a782c3e43c94067a571c2342bf92165": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_55ce1d040d1d496ca8d15e8e67c74555",
      "max": 1042301,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_98fe1d638a9c47a3a29217dec010271f",
      "value": 1042301
     }
    },
    "4ad1469d6645401f9f8d52149fd2d03a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "4b2a26e21c8548f29fe0cde2b1159dc4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_af11a4a4d46d4f20825343a155cc5a50",
      "placeholder": "​",
      "style": "IPY_MODEL_6b1a6e616e9e4664952e3594af71ae40",
      "value": "tokenizer.json: 100%"
     }
    },
    "4fddf09bfffa4b379a7a6378d9856b20": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "50b466e09a1544c595221e94d08d68cf": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "5115733ac8814b1b8eaf30d28dd62d15": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "51425daae10f433987192f14faedde7b": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "51c3e0e8584340a2ba1482adbd59cb06": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "526dc2e823a74e71a84cb4b0d50cd5be": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "534ac470d5974a57bdc7dd56177baea2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_7fb83f1ddbdd4223aadc1cec6960f4d9",
       "IPY_MODEL_0f5b75071a944a7490e2aa111c4f79f9",
       "IPY_MODEL_5834698eed4643a59c6092665a26e7c6"
      ],
      "layout": "IPY_MODEL_2cce25a349ce479fbb0c8ee33c88eb6c"
     }
    },
    "5397dbf6312c400c816a8de45c7144a6": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "53c80c8b1f9d4a1f9607be86275abe77": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "54a77ee7423f4d649a371bc4a2909b60": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "54c90b8fd8854de289642a284d145e29": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "5520bf0222074846a5ca67d326a68d0a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "55ce1d040d1d496ca8d15e8e67c74555": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "56c8efa2aff44f90a1a595783d3d7d1c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "579ccf522ee643e188f9c975d96b5e63": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "5834698eed4643a59c6092665a26e7c6": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_dae1a036532f45b199f6cd3a703dfbaa",
      "placeholder": "​",
      "style": "IPY_MODEL_db536d7ed52d4037b3aa89d8acb2c999",
      "value": " 333/333 [00:00&lt;00:00, 21.8kB/s]"
     }
    },
    "585aa26fe56342aaa4b17d4d2876420e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": "20px"
     }
    },
    "58d3dc01f23243448e73ae9dfaa32301": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "5907f9914df3490da8af4435fdd47471": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "59382ec30b664f0cb69bd1ee669a3412": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_f4c605241d9843118aa091cb01063476",
       "IPY_MODEL_d3fe49448a7f4d28bc0d7c3d60596989",
       "IPY_MODEL_feb5600fce7445b6b6454a4aeb072e13"
      ],
      "layout": "IPY_MODEL_9a731e2a431a4a039a7f9c7974c6dddc"
     }
    },
    "5950da05ca49401d9a8ec085b5e54fa8": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "5b0ee610309d47088af9a940d901038a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_ca28a78a4fcf41919f03fab4488e2f10",
       "IPY_MODEL_42cdd539bdcd4563ac7263ca3a87a795",
       "IPY_MODEL_e16b3717548e405ea82b33e6f1ed9aeb"
      ],
      "layout": "IPY_MODEL_7ff2fe616e7f4890a3d7e5dd5b1539bf"
     }
    },
    "5c191aa20c87406f9b7e56c286d014f6": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "5c3310bb65de4d178cc9a999a55185e1": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "5cd71a878a2b45a9bdaaf56f32f2da6c": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "5d75918be7df4027a56f972503bd8b78": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "5ecff38cd27f4a1ba36dfaddaaee0bb9": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "5f54bf184d3d4e8687543ecd27afc3cc": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "5f8193d2901645feb4f3fc80d4942bad": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "601f9cc9a5794be48c4df19677e3ba27": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "60b5f4007d9640279a5de2949625fb2b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "61ee1b029cd348ecaa0cd6439ff904fc": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "62643405eecf4af2bad191b99bffd84c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_6f34c93d9c1a45d5a9abe74c5fa4f365",
      "placeholder": "​",
      "style": "IPY_MODEL_a821a5bb13444450b3bd785f20dfc421",
      "value": "Generating unsupervised split: 100%"
     }
    },
    "649625e0f9be4c9ebb0a119076333197": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_26438f4a85cb4e05923cbd1444df2364",
       "IPY_MODEL_2cab9533d3c1442a90872341972df272",
       "IPY_MODEL_1c5621316bce482abcd926e7a053a9b0"
      ],
      "layout": "IPY_MODEL_5cd71a878a2b45a9bdaaf56f32f2da6c"
     }
    },
    "64bb37c8f5194853a8da7a149f0f6133": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "6506e794db9b41a8822f1e6744031979": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "656bbd9009654ed6af0c61dba2f01d4d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "66eaee16e9ec490086dffd09369ccf60": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "67e45ac4538d4183b1eea084e899dd48": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "680643b191104f38adcb2aef4cf7d5c1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_5520bf0222074846a5ca67d326a68d0a",
      "placeholder": "​",
      "style": "IPY_MODEL_a12bebd26bb946c59549a703b98c3878",
      "value": " 50000/50000 [00:00&lt;00:00, 155508.98 examples/s]"
     }
    },
    "6831d41e645b4181b03c7f61d0205211": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "68fd2d482bdc40a38bd1d6bbeef7b8e0": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_1dd60f1ebabe4d74994a51b688c73b80",
      "max": 20470363,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_54c90b8fd8854de289642a284d145e29",
      "value": 20470363
     }
    },
    "6960797f105246e582de14c69bd88171": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e106952a01f54610997d830e471ebda0",
      "placeholder": "​",
      "style": "IPY_MODEL_9cde8a8dcafe4237bfb5e96075b9f158",
      "value": "<center> <img\nsrc=https://huggingface.co/front/assets/huggingface_logo-noborder.svg\nalt='Hugging Face'> <br> Copy a token from <a\nhref=\"https://huggingface.co/settings/tokens\" target=\"_blank\">your Hugging Face\ntokens page</a> and paste it below. <br> Immediately click login after copying\nyour token or it might be stored in plain text in this notebook file. </center>"
     }
    },
    "6a106ae702b546538de0429353d4ba62": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ButtonModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ButtonModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ButtonView",
      "button_style": "",
      "description": "Login",
      "disabled": false,
      "icon": "",
      "layout": "IPY_MODEL_526dc2e823a74e71a84cb4b0d50cd5be",
      "style": "IPY_MODEL_29ce0e9095b646c3b50d721559af6bf6",
      "tooltip": ""
     }
    },
    "6b1a6e616e9e4664952e3594af71ae40": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "6b22ba5fc87440c1a19fb4ba3f0659ad": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "6d56b2617f0348eeb2d7d8a109646814": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_017d20c252904df1bdb48bda9c8c1465",
       "IPY_MODEL_274fa2d33c0247c1bb584751a64e28a7",
       "IPY_MODEL_3b1c47003ad54aeaa3bfeae65a71ffc6"
      ],
      "layout": "IPY_MODEL_ff50bca48f884f9b841a4881e1083b24"
     }
    },
    "6f34c93d9c1a45d5a9abe74c5fa4f365": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "728d583ea43d45b9a002866a0a857871": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "731e42b4b9e3444eadff0c496976389e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "LabelModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "LabelModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "LabelView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d39ee62683574bd0a02930bef6b94dcc",
      "placeholder": "​",
      "style": "IPY_MODEL_47c77d548220417abec9f72df138401a",
      "value": "Token is valid (permission: fineGrained)."
     }
    },
    "73ed87030dc24eb681d9ba9d0f4b9f11": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": "center",
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": "flex",
      "flex": null,
      "flex_flow": "column",
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": "50%"
     }
    },
    "74766eb042fc48c1b8fd5cab9143f5b1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "75ada9f5c4d9466e8a42597ea9ac6466": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_0de62df79d03478184c5c5413795e479",
      "placeholder": "​",
      "style": "IPY_MODEL_3c426c71a2d147c1af9330ce0703e14d",
      "value": " 665/665 [00:00&lt;00:00, 22.4kB/s]"
     }
    },
    "7ac268d459dd4ceb98237057ef63b5cf": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "7b6bbb45ed2d4f77a848b6ba29d2f23b": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "7fb83f1ddbdd4223aadc1cec6960f4d9": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_040ec7d31dc24b269cc6593d98d26fe8",
      "placeholder": "​",
      "style": "IPY_MODEL_74766eb042fc48c1b8fd5cab9143f5b1",
      "value": "tokenizer_config.json: 100%"
     }
    },
    "7fee544e10ba4b399bae8c1a68c16e5b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "7ff2fe616e7f4890a3d7e5dd5b1539bf": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "803e811bc7a14030a80366cdc714b9fa": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "806a776295c44eb7998dca7ace00c6cf": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "80b0498891874b709d8291a47fe70350": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "CheckboxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "CheckboxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "CheckboxView",
      "description": "Add token as git credential?",
      "description_tooltip": null,
      "disabled": false,
      "indent": true,
      "layout": "IPY_MODEL_addcfa65505e4679a03c2ef1ceedf8c0",
      "style": "IPY_MODEL_f24072cbb64e4011af425afb51adabfb",
      "value": true
     }
    },
    "80f799821f6145238fccc14826e56e8e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "816feb637b724bca83ce7347d0b94547": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "82554e9776da462b9dd76d3393d67871": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_cfcb16d2c2464e5cb52d55258ac22249",
       "IPY_MODEL_bde58246b1bf4e2facd5bd24ec228783",
       "IPY_MODEL_124fa02529c7427593b3bf71d4612646"
      ],
      "layout": "IPY_MODEL_51425daae10f433987192f14faedde7b"
     }
    },
    "830bc949b24f49a49cc4fdd8de2db021": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_579ccf522ee643e188f9c975d96b5e63",
      "placeholder": "​",
      "style": "IPY_MODEL_aec19b7c46244887bb4be678639acb2c",
      "value": "config.json: 100%"
     }
    },
    "843c3e08d63547d9a42eec08ecd8f1ed": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_5950da05ca49401d9a8ec085b5e54fa8",
      "placeholder": "​",
      "style": "IPY_MODEL_4fddf09bfffa4b379a7a6378d9856b20",
      "value": "vocab.json: 100%"
     }
    },
    "8529ec8a1e594b84aca64f9ba6b5e5e0": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_dddb7ffab70e419ebe771557b84421d6",
      "placeholder": "​",
      "style": "IPY_MODEL_0ba241098153433eb162fa4dcac0e6bc",
      "value": " 112/112 [00:00&lt;00:00, 8.24kB/s]"
     }
    },
    "8563d658a4834e16a59636d8d6b1c512": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_d8804046759d46c89f9a5cd2512d5b7a",
       "IPY_MODEL_ed38c5091aad4446ab34b82f318531d7",
       "IPY_MODEL_4341e013b9dd4c87ac30d7e74978834f"
      ],
      "layout": "IPY_MODEL_0edf9f054c3c4e18976736d908a803f5"
     }
    },
    "875ffc1310844528b287d4a257cf5f97": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8a12f57eccaa4878a5f7d4f64234f02b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_5115733ac8814b1b8eaf30d28dd62d15",
      "max": 50000,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_9d958d164c674abe9980ed3d2d5bdd9b",
      "value": 50000
     }
    },
    "8a7ecf987e09457f97a12db63998ed09": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_843c3e08d63547d9a42eec08ecd8f1ed",
       "IPY_MODEL_4a782c3e43c94067a571c2342bf92165",
       "IPY_MODEL_b2012e7cfe3e408e82a4cf52d4ff06bf"
      ],
      "layout": "IPY_MODEL_d530e3d8282e4971a94dfbbd47a4f0ff"
     }
    },
    "8c3269edbb0148259348b37cfd2ee694": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e51622ab9b1a429799f6e823a41e58db",
      "max": 466132,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_24d1ae4c531f44549a69cbf13435b155",
      "value": 466132
     }
    },
    "8ca44fcac9be4aaab123128bd6743e68": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_059b7ac7e5e6451097d783e404cf7a7e",
      "placeholder": "​",
      "style": "IPY_MODEL_df96c55c357b481fab626a687e44c6e0",
      "value": "train-00000-of-00001.parquet: 100%"
     }
    },
    "8de59c241b574fcb83b8036ca44fef17": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8e3691cfb96044e5bb17483a69b80b3a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "901639f4ef874a529c6a194a46fa5f32": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_0a70d9f2928841a5ab5a71a1f90beca6",
      "placeholder": "​",
      "style": "IPY_MODEL_58d3dc01f23243448e73ae9dfaa32301",
      "value": "Filter: 100%"
     }
    },
    "905482147fcc4b0eadad09ec99333097": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_af13120b305c49ddb60dec814bb23ad0",
      "placeholder": "​",
      "style": "IPY_MODEL_2fb4cb7c8c6f44a2a5cc4f8badde0b9a",
      "value": "config.json: 100%"
     }
    },
    "917afc28d0ae448c95cadf2d2cd9eeab": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "91f8f77b67f34b6b8744b6018c3f2ed3": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9338547fd19542ac847d2e9ee95244a1": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "945a08de02ee4bef9ecd00e64b582246": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f7aeb464ce3042e2b0df49978a777a8b",
      "placeholder": "​",
      "style": "IPY_MODEL_fedfb03cd8b84712b2b10636dc298178",
      "value": " 26.0/26.0 [00:00&lt;00:00, 611B/s]"
     }
    },
    "94817bcac7374e43b4b633e0f2d13889": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "LabelModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "LabelModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "LabelView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e23b1364318a46f680cabedc7f0c6d30",
      "placeholder": "​",
      "style": "IPY_MODEL_08ffbe2a042544aeb8565179a327c986",
      "value": "Your token has been saved in your configured git credential helpers (store)."
     }
    },
    "9561bbe794774920a2594c87cc2822b7": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "96f0ee289dd54e89861f8977d883b201": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "97412f765b2942519106c00ffa2020ba": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "98fe1d638a9c47a3a29217dec010271f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "9a731e2a431a4a039a7f9c7974c6dddc": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9bfe331c17c546a6ac58a1bb2fa874c7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_ff87ca71563b4cdb8d2cbf4c690c1922",
       "IPY_MODEL_1f1f3151f6cf4b3da646f13253f5fbb1",
       "IPY_MODEL_408b38405e21448fb939f2609e8c2cea"
      ],
      "layout": "IPY_MODEL_ed4a5a90856340ae94db47241ac7e14a"
     }
    },
    "9c880f7546e64072ab12908be8a9da94": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_007d96ccd3ec436ab7b4e53589df8171",
       "IPY_MODEL_096e494497254950b4c4eb2af68df123",
       "IPY_MODEL_190f641967bb4bad9706a642ac7c2da2"
      ],
      "layout": "IPY_MODEL_803e811bc7a14030a80366cdc714b9fa"
     }
    },
    "9cde8a8dcafe4237bfb5e96075b9f158": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "9d464f2073fb4b5e9f3e695314a4b082": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_20ba4fcb1fe0441d9f8250904390a1bf",
       "IPY_MODEL_f647c4105f9a4a6f9d749647eeae6526",
       "IPY_MODEL_ff95bf3112384668bc376070c4cda552"
      ],
      "layout": "IPY_MODEL_d814ab71fbc44031a6b132e2af8d7f0e"
     }
    },
    "9d5becf11dbe4b22a706147b799f2d51": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "9d958d164c674abe9980ed3d2d5bdd9b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "9dc38fd046024f95b711cc476cc138d8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "9e51a881928740c79ed471587b9f4e98": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "9ebfe425886c46958067b0ce4aac1980": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a09a7a2ff8804931b7ad83ee773e0417": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a12bebd26bb946c59549a703b98c3878": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "a1334367870e4c27b47d8638bacd1c8f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_da0598234e484de9872383106d5b45e2",
       "IPY_MODEL_2645f71273704bc1823828adf4e31f9d",
       "IPY_MODEL_16656c4fc26f48c3bbbcbbb94bbfd5c3"
      ],
      "layout": "IPY_MODEL_a9d5de9af25048dda9553df10cf52ac9"
     }
    },
    "a1794db67fb74f7a9393b88c11ccf57b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_53c80c8b1f9d4a1f9607be86275abe77",
      "max": 25000,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_d17b23a11a7c4e6db11c049bf3d4d69f",
      "value": 25000
     }
    },
    "a32361ab80ce4528907e7969a762075d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a821a5bb13444450b3bd785f20dfc421": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "a9d5de9af25048dda9553df10cf52ac9": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ab5e76fdf7d5439c8c0ee63a126923b2": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ab8ecce0350447e9898cd288be6654dd": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "addcfa65505e4679a03c2ef1ceedf8c0": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "aec19b7c46244887bb4be678639acb2c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "af11a4a4d46d4f20825343a155cc5a50": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "af13120b305c49ddb60dec814bb23ad0": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b2012e7cfe3e408e82a4cf52d4ff06bf": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_c8a1a90eb49e487cba872d50213fe0ac",
      "placeholder": "​",
      "style": "IPY_MODEL_1b9ef18c0fbf4ff39d74e2d54c107991",
      "value": " 1.04M/1.04M [00:00&lt;00:00, 3.23MB/s]"
     }
    },
    "b65c9e0f00be4ff387b52094d11259ed": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_02538b35512e40cc90c7da5c44d439f0",
      "max": 41996509,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_4ad1469d6645401f9f8d52149fd2d03a",
      "value": 41996509
     }
    },
    "b75b711dc88f43f9b757697de1bdcf90": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_5c3310bb65de4d178cc9a999a55185e1",
      "placeholder": "​",
      "style": "IPY_MODEL_9dc38fd046024f95b711cc476cc138d8",
      "value": " 21.0M/21.0M [00:00&lt;00:00, 63.8MB/s]"
     }
    },
    "b7ae5d5cecee4485b7aaa6fe7571182c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "VBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "VBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "VBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_731e42b4b9e3444eadff0c496976389e",
       "IPY_MODEL_94817bcac7374e43b4b633e0f2d13889",
       "IPY_MODEL_dfd2fad3bba04fd295ee408c91869f7c",
       "IPY_MODEL_e84412f70dd049dea7cb079c2c11c3d4"
      ],
      "layout": "IPY_MODEL_73ed87030dc24eb681d9ba9d0f4b9f11"
     }
    },
    "bd2c9ebf67a04a538f3bde44ce8107f3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_6b22ba5fc87440c1a19fb4ba3f0659ad",
      "max": 20979968,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_0c73a3203dff4a949e7c15be100277c5",
      "value": 20979968
     }
    },
    "bd5a32bb27234df091bf15edd8859c00": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_96f0ee289dd54e89861f8977d883b201",
      "placeholder": "​",
      "style": "IPY_MODEL_369fac0bbb324623acc8f2a9ad3c211e",
      "value": " 25000/25000 [00:00&lt;00:00, 191060.09 examples/s]"
     }
    },
    "bde58246b1bf4e2facd5bd24ec228783": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_45ca8ac26b964d05bb5e42cdd31a012a",
      "max": 25000,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_fe307220f8124c888337c0f191186253",
      "value": 25000
     }
    },
    "c217140ac4a64713bb1fe265d01782cd": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "c3f17dc960794f2385943498969978e2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "c47e110edc9943a99a2b02bdb1d1087d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_830bc949b24f49a49cc4fdd8de2db021",
       "IPY_MODEL_11ce622e2580467d807fa8e7dd87e3b5",
       "IPY_MODEL_49b4cd3787ef47399b90c14a627a67da"
      ],
      "layout": "IPY_MODEL_9338547fd19542ac847d2e9ee95244a1"
     }
    },
    "c84eb5f041614dc4bf577a45edc3b26d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c898ec71d4434765a94670abd972edb2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "c8a1a90eb49e487cba872d50213fe0ac": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c943662f45a34434b2c047a5d1019e04": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "LabelModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "LabelModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "LabelView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_656bbd9009654ed6af0c61dba2f01d4d",
      "placeholder": "​",
      "style": "IPY_MODEL_5ecff38cd27f4a1ba36dfaddaaee0bb9",
      "value": "Connecting..."
     }
    },
    "ca28a78a4fcf41919f03fab4488e2f10": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_da3a18c49dc54741a40ba5aa204b39b9",
      "placeholder": "​",
      "style": "IPY_MODEL_09e2170d4b814dec842545ffef6aa186",
      "value": "merges.txt: 100%"
     }
    },
    "ca8c415219f4467b92c2d06ffa182c67": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "cae1ddea6b0b4accb7a5ea9a320070ea": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "cf06e4b58d0949cfac2c5025dc68d014": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "cf0f4ffec6604e9ca6fc9667ef4b32e2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_1cb383c5a22c46d5908be7c285f394f3",
      "placeholder": "​",
      "style": "IPY_MODEL_ddaf8d2bf0fe40e8a1278017b76d0d11",
      "value": " 5.17k/5.17k [00:00&lt;00:00, 244kB/s]"
     }
    },
    "cfcb16d2c2464e5cb52d55258ac22249": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_64bb37c8f5194853a8da7a149f0f6133",
      "placeholder": "​",
      "style": "IPY_MODEL_c3f17dc960794f2385943498969978e2",
      "value": "Generating test split: 100%"
     }
    },
    "d1269dd6eb1e45b1aa1a6589dc9f642f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "d17b23a11a7c4e6db11c049bf3d4d69f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "d297e0a54a8e4afe90d883ce17e22c05": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "d384ee5f00a6468489506ac470be90fa": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_2bc64794e64d4cdfbd5222651a469857",
       "IPY_MODEL_b65c9e0f00be4ff387b52094d11259ed",
       "IPY_MODEL_493432db1fb84848964d14966627fc4b"
      ],
      "layout": "IPY_MODEL_30844b55fec7479d82802a69b036eb1f"
     }
    },
    "d39ee62683574bd0a02930bef6b94dcc": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d3fe49448a7f4d28bc0d7c3d60596989": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_490eddc9ab7048e8bfdc15ba8f5fa6a7",
      "max": 267860081,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_d5736bbe814144878a30837e2efba596",
      "value": 267860081
     }
    },
    "d42211410ee64e7b9fff9c3653dea5b0": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "d4f74fade7c94c2c88911394fe3f78dd": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_0c75945a099a4f6a850a2cdedfbc1d23",
      "max": 112,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_c898ec71d4434765a94670abd972edb2",
      "value": 112
     }
    },
    "d530e3d8282e4971a94dfbbd47a4f0ff": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d5736bbe814144878a30837e2efba596": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "d814ab71fbc44031a6b132e2af8d7f0e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d8804046759d46c89f9a5cd2512d5b7a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_61ee1b029cd348ecaa0cd6439ff904fc",
      "placeholder": "​",
      "style": "IPY_MODEL_d297e0a54a8e4afe90d883ce17e22c05",
      "value": "Generating train split: 100%"
     }
    },
    "d9bd270f150c4e8fab043cc2fe0478f3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "d9e2f543e5544f85b24fff37f375ae88": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f3b3939fded04c1397ebb11010abd533",
      "max": 26,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_56c8efa2aff44f90a1a595783d3d7d1c",
      "value": 26
     }
    },
    "da0598234e484de9872383106d5b45e2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_7ac268d459dd4ceb98237057ef63b5cf",
      "placeholder": "​",
      "style": "IPY_MODEL_ab8ecce0350447e9898cd288be6654dd",
      "value": "model.safetensors: 100%"
     }
    },
    "da3a18c49dc54741a40ba5aa204b39b9": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "dae1a036532f45b199f6cd3a703dfbaa": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "db536d7ed52d4037b3aa89d8acb2c999": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "dc0b9472eea14c7bbd285bef3d044583": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "dcb3e035278646df81517bcbd63ad671": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "ddaf8d2bf0fe40e8a1278017b76d0d11": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "dddb7ffab70e419ebe771557b84421d6": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "df2678cc4bf74b01b5e0d7a223005d9e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_3001b740eca44b0b9741b3a4e4c4de0d",
      "placeholder": "​",
      "style": "IPY_MODEL_60b5f4007d9640279a5de2949625fb2b",
      "value": "README.md: 100%"
     }
    },
    "df96c55c357b481fab626a687e44c6e0": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "dfd2fad3bba04fd295ee408c91869f7c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "LabelModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "LabelModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "LabelView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_42f8b2a5cc8845c489dadaade07e35df",
      "placeholder": "​",
      "style": "IPY_MODEL_17c551db5ae944a1ad9fe7fc38f88a9f",
      "value": "Your token has been saved to /root/.cache/huggingface/token"
     }
    },
    "e106952a01f54610997d830e471ebda0": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e16b3717548e405ea82b33e6f1ed9aeb": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_1f038ef1ec8b4d1e88654b68178406f4",
      "placeholder": "​",
      "style": "IPY_MODEL_21168c6da7434941a8062e727a8acb74",
      "value": " 456k/456k [00:00&lt;00:00, 1.88MB/s]"
     }
    },
    "e23b1364318a46f680cabedc7f0c6d30": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e454660115b240f1aea38f25d04012e0": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e51622ab9b1a429799f6e823a41e58db": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e55a650c0f6f48fdb643ac0c3ecb203f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "e76ba5f1a6ce49229bee1f51679d1efc": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e8022095601b4285aad6dd1a7181cd6d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e84412f70dd049dea7cb079c2c11c3d4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "LabelModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "LabelModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "LabelView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_fbe86cfbd1d64b02b43adfd5304bca49",
      "placeholder": "​",
      "style": "IPY_MODEL_d1269dd6eb1e45b1aa1a6589dc9f642f",
      "value": "Login successful"
     }
    },
    "ea257116758f400da8751eef1886bbea": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "eac93a970ece49f1b43969cde67d6619": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "eca055c6a073422dbc6038850d90166d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ed38c5091aad4446ab34b82f318531d7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_728d583ea43d45b9a002866a0a857871",
      "max": 25000,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_3f17fc1b5c2c4f1996151cbded670ae3",
      "value": 25000
     }
    },
    "ed4a5a90856340ae94db47241ac7e14a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ee10b481e2ee44f4abf32639dbbd1919": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f12bf5040447467a882e40fcf3e972d4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_f2750715312a40b0ab1ba8c2343f9a98",
       "IPY_MODEL_d9e2f543e5544f85b24fff37f375ae88",
       "IPY_MODEL_945a08de02ee4bef9ecd00e64b582246"
      ],
      "layout": "IPY_MODEL_5f8193d2901645feb4f3fc80d4942bad"
     }
    },
    "f24072cbb64e4011af425afb51adabfb": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "f260f5c57a2948ca82c3862639405f13": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "f2750715312a40b0ab1ba8c2343f9a98": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e8022095601b4285aad6dd1a7181cd6d",
      "placeholder": "​",
      "style": "IPY_MODEL_2ac034bd2c6d44d6aad88f8af57d3f13",
      "value": "tokenizer_config.json: 100%"
     }
    },
    "f2a33680c4ce4e8face4bbd17b2fec03": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_51c3e0e8584340a2ba1482adbd59cb06",
      "placeholder": "​",
      "style": "IPY_MODEL_c217140ac4a64713bb1fe265d01782cd",
      "value": "special_tokens_map.json: 100%"
     }
    },
    "f2ca9a2ef8d34ffc9d5ef1923225f970": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "f3b3939fded04c1397ebb11010abd533": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f459c938c12542c3837b019c0770a760": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f4c605241d9843118aa091cb01063476": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_917afc28d0ae448c95cadf2d2cd9eeab",
      "placeholder": "​",
      "style": "IPY_MODEL_cf06e4b58d0949cfac2c5025dc68d014",
      "value": "pytorch_model.bin: 100%"
     }
    },
    "f4d8ccd596bd4b30bbbf2e5cf80c35f5": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f57de9509e874cb38ef90cfc2e266a2f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_2bf0912777294d7384ac246fca984f94",
       "IPY_MODEL_68fd2d482bdc40a38bd1d6bbeef7b8e0",
       "IPY_MODEL_3758d7613cca4c6b9309e7e991cf03f5"
      ],
      "layout": "IPY_MODEL_3bd73db8e99a42c2ad84ec04f32e8113"
     }
    },
    "f5a4aa9b9275450e8d3f028b5ee4d4ce": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "f647c4105f9a4a6f9d749647eeae6526": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "danger",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_585aa26fe56342aaa4b17d4d2876420e",
      "max": 1,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_8e3691cfb96044e5bb17483a69b80b3a",
      "value": 1
     }
    },
    "f7037dc6dd074cc483e6482bea4581c1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "f7aeb464ce3042e2b0df49978a777a8b": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "fa2b4efbbfe0442580a85b0efae75d90": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "fbe86cfbd1d64b02b43adfd5304bca49": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "fe13e5e8b3d5440eb680e36f3ac6107d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_901639f4ef874a529c6a194a46fa5f32",
       "IPY_MODEL_a1794db67fb74f7a9393b88c11ccf57b",
       "IPY_MODEL_bd5a32bb27234df091bf15edd8859c00"
      ],
      "layout": "IPY_MODEL_f4d8ccd596bd4b30bbbf2e5cf80c35f5"
     }
    },
    "fe307220f8124c888337c0f191186253": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "feb5600fce7445b6b6454a4aeb072e13": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a32361ab80ce4528907e7969a762075d",
      "placeholder": "​",
      "style": "IPY_MODEL_f2ca9a2ef8d34ffc9d5ef1923225f970",
      "value": " 268M/268M [00:00&lt;00:00, 291MB/s]"
     }
    },
    "fedfb03cd8b84712b2b10636dc298178": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "ff50bca48f884f9b841a4881e1083b24": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ff87ca71563b4cdb8d2cbf4c690c1922": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_3e7a016083d44af6a1d8b047b803cdb0",
      "placeholder": "​",
      "style": "IPY_MODEL_9e51a881928740c79ed471587b9f4e98",
      "value": "README.md: 100%"
     }
    },
    "ff95bf3112384668bc376070c4cda552": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ab5e76fdf7d5439c8c0ee63a126923b2",
      "placeholder": "​",
      "style": "IPY_MODEL_6831d41e645b4181b03c7f61d0205211",
      "value": " 5/? [02:15&lt;00:00, 27.00s/it]"
     }
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
